Skip to content

Tags: alpha123/zambesii

Tags

v0.00.013

Toggle v0.00.013's commit message
Bumping the kernel to v0.00.013, citing the achievement of a major mi…

…lestone:

We have UDI drivers executing, channels working, calls in and out of drivers
working, and the initialization sequence genericized and repeatable. Apart from
that, this version increment sees the addition of many beneficial changes to
the kernel.

* Uniform Driver Interface:
  * Added several metalanguages to the kernel tree:
    ^ udi_gio
    ^ udi_physio
    ^ udi_mgmt
    ^ zbz_root.
  * Introduced libzbzcore and libzudi.
  * Several syscalls for UDI implemented, including udi_mei_call and the
    udi_channel_* functions.
  * Implemented UDI channels.
    ^ Bind channels and normal channels.
  * Implemented UDI regions.
  * UDI drivers now mostly work.
  * The kernel now supports multi-parent devices, due to the fact that I now
    properly understand how udi_enumerate_ack() provides a metalanguage for
    child devices that it enumerates (through the ops_idx argument).

* Cleaned up the VFS base types, added PATH_MAXLEN macros.
* Migrated from the ZMESSAGE infrastructure to a new, cleaner, centralized,
  and better designed grand-central message queueing system, ::MessageStream.
  * Renamed subsystem macros accordingly.
  * Renamed function macros accordingly.
  * New user-queues for posting messages between threads in the same process.
    ^ Accompanying API is the MessageStream::postMessage() method.
  * Introduces the Lock::sOperationDescriptor struct, which is used to remove
    the threat of lost wakeups.
* New kernel "IPC" class, for inter-thread communication: ZAsyncStream.
  ^ Enables a process to listen for connections and nominate a thread to handle
    requests over each new connection request.
  ^ Also has an "accept all" mode which allows messages to be taken without
    a pre-established connection. Useful for kernel server threads.
* Style changes:
  * Renamed all classes in the kernel from myClassC to MyClass.
  * Renamed all structs in the kernel from myStructS to sMyStruct.
  * Renamed __KNULL to NULL.
  * Renamed __kprintf to printf.
* New types for asynchronous programming assistance.
  ^ ::Callback, template<> ::_Callback, ::__kCallback.
  ^ AsyncResponse.
* Floodplainn:
  * The birth of Floodplainn itself.
  * Implemented the Floodplainn VFS.
  * Implemented device types:
    * fplainn::Driver.
    * fplainn::Device.
    * fplainn::DeviceInstance.
    * fplainn::DriverInstance.
  * Devices that share the same driver will exist in the same address space.
    * Can now spawn driver processes.
  * Root virtual bus layout decided.
  * Connect() and Send() APIs allow normal processes to directly connect to
    a running device instance.
  * Added an array of mapping entries that translate udi metalanguage names into
    device class names.
* Birth of the ZUI Server:
  * Created the ZUDI repository.
  * Created the ZUI server.
  * Implemented the ZUI parser classes.
  * Implemented the ZUI Server's driver matching code, as well as driver
    metadata loading, etc:
    ^ detectDriver, loadDriver, spawnDriver, instantiateDevice.
  * ZUI UDIProps database is now an integral part of the kernel build process.
* Birth of the ZUM Server:
  * This is essentially the Zambesii UDI Management Agent.
    ^ Makes calls into drivers' MA channel on behalf of the kernel.
    ^ Sends udi_channel_event_ind calls into drivers on behalf of the kernel.
* Lib and utility:
  * Printf has a recursive format string parser (%r), and has been somewhat
    reworked to be more decent.
  * Most list classes now have Iterators.
  * New RAII classes for heap objects.
  * Fixed serious errors in our implementation of operators new[] and delete[]
    due to lack of understanding.
  * The kernel has a new heap!
    ^ This new heap should be able to support realloc().
    ^ It is an aggressive, self-analyzing monster that detects bugs.
    ^ Has exposed, single-handedly a whole lot of bugs.
    ^ Supports guard pages.
  * We now have stack traces!
* General:
  * Kernel can now sort of build in Clang++.
  * The kernel now no longer supports response message redirection of kernel
    API response messages.
  * Namespace ipc:: exports a series of mapping and IPC data transfer methods
    for kernel use elsewhere.
  * On x86-32, demand mapping will try a max of 4 times for pmem.
  * The kernel no longer splits up threads using the nasty, bizzare scheme that
    we had in place before. We now have a power thread per CPU, and each CPU
    has its own power stack.
  * We now have early debug output, and a new kernel boot sequence which is far
    more micro-kernel-esque.
  * Kernel is now built with -fno-omit-frame-pointer.

v0.00.012

Toggle v0.00.012's commit message
Version increment to v0.00.012, as process spawning is mostly complet…

…e. Moving

on to the completion of the storage VFS next after some cleanup.

This version increment sees a good few interesting features introduced,
including but not limited to:

* Process spawning.
* Per-CPU Threads.
* processStreamC, containerProcessC, containedProcessC, distributaryProcessC are
  all introduced by this version increment.
* Memory Tributary now accepts a NO_SLEEP flag.
* Kernel now properly handles top-level page table entry changes and propagates
  them across processes as they are pulled by the scheduler.
* Updated the notes for x86-32/IBM-PC's placement addresses.
* Unified and cleaned up the way that the kernel process is initialized.
* Kernel process now has ID 0x10000, and process ID 0 is reserved for per-cpu
  threads.
* Processes now have command lines, arguments, and environment variables.
* Implemented a uniform callback infrastructure for threads to pull messages off
  of.
* The kernel now uses the freestanding headers of the compiler being used to
  build it.
* The behaviour of the "ID" member of processes and threads is slightly
  different.
* Added basic support for LLVM. LLVM can be used to build the kernel, but the
  output binary does not run correctly.
* Removed __kequalizerListC from use within the kernel vswamps, replacing it
  with the new infrastructure in slamCacheC.
* Updated kernel design docs.
* Distributary Trib processes can be executed now.
* Changed the kernel's NOTICE/WARNING/ERROR/FATAL prefix strings; now they are
  shorter.
* Fixed virtual memory leaks in libx86MP and libACPI.
* taskTribC::block() now can take a lockC* argument and unlock this object
  after modifying the target CPU's schedqueue. This change is part of the
  solution to lost-wakeups from waiting on a single queue. A solution for
  threads waiting on multiple queues is yet to be found.
* The stack structure of CPUs is now different: CPUs now have a per-cpu thread
  stack and a schedstack instead of a single sleepstack.

v0.00.011

Toggle v0.00.011's commit message
Version increment to v0.00.010. The next version increment is far more

ambitious: will be hopefully incrementing to version 0.01.000. This version's
main feature debut is the Timer Tributary and the Timer Streams. They together
provide the kernel's timer API.

* Began adding support for Clang to the build system.
* Kernel now officially supports only C++03. No C++11, etc.
* Loading bus-pin mappings for a bus will also cause the bus-pin mapping code
  to program the IRQ controller pins for that bus.
* Created a platform specific placementAddresses.txt log.
* Kernel boots all AP CPUs again now.
  ^ It does this in parallel too, using asynchronous timer events.
* Task Tributary:
  * Fixed bugs in saveContextAndCallPull() and loadContextAndJump().
  * Implemented basic scheduling functions such as yield(), dormant(), wake(),
    block(), unblock().
* __kclasses:
  * Implemented singleWaiterQueueC, a simple wait queue.
    ^ Has support for non-blocking pop().
* IBM-PC:
  * i8254: A good few refinements.
    ^ Holds an internal singleWaiterQueueC object for IRQ events.
* Timer Tributary:
  * Defined the structure and code for the event dequeuer thread.
    ^ Checks multiple queues, with a dedicated, prioritized message queue for
      control messages from the kernel.
    ^ Should be reasonably free of races.
  * Defined the prototypes for the timer API.
  * Defined the layout of the Timer Stream event objects.
  * Added support for allowing the chipset to hook a clock callback routine into
    a timer Queue to keep the system clock updated. Will probably be obsoleted
    soon enough.
* Process/Thread:
  * Each thread now has its own event queue for each event.
  * Moved the pendingEvents BMP out of taskC into processStreamC.
  * Threads are no longer spawned with local IRQs disabled by default.
* CPU Tributary:
  * InterCpuMessagerC: Got a moderate overhaul of its state machine.
    ^ No longer controls local IRQ acceptance for its parent CPU.
  * Added powerManagerC to each CPU.
  * Created an Object Oriented class for LAPICs on x86.
* Interrupt Tributary:
  * Re-structured the way interrupts are permeated to the handling logic.
    * Now split into pinIrqMain(), exceptionMain(), swiMain() msiIrqMain().
    * Pin IRQs now have a new ZKCM ISR exit status:
      ZKCM_ISR_SUCCESS_AND_RETIRE_ME, which removes the ISR after executing it.
      Fixes a race condition.
* x86 (-32):
  * Implemented basic support for the Debug Registers.

v0.00.010

Toggle v0.00.010's commit message
Added a license to the kernel tree. The kernel is now officially GPLv2

licensed. Also added a readme with information on the commercial intent
for the kernel.

This version increment also introduces changes to the build system that
include a smooth, ./configure && make style build, with an interactive
configure that can be extended to support multiple build toolchains.

A lot of the work done in this version increment has to do with the
kernel's time types, the timer subsystem, and scheduling since this was
the core goal of this next version. Specific notable time types
implemented are timeS, date_t and timestampS. The kernel takes time
to be gregorian, and userspace libraries are used to convert into
POSIX epocs. The kernel now also takes note of the date and time it was
booted (boot timestamp).

The ZKCM Driver subsystem and APIs have been revamped to be object
oriented, extensible, and to be somewhat uniform with the UDI model,
enabling them to co-exist synergetically with the soon-to-be-implemented
UDI environment.

Various minor bugs were fixed, and I also added NULL pointer checking to
the functions in the C string lib. Some language level improvements have
also been made: almost all HLL files in the kernel are now C++ and not
C. Additionally, I began "C++izing" the kernel a bit.

Chipset related improvements include a more sane chipset SMP mode
switch, better detection of bad SMP boards, and the implementation of
a very important chipset specific API, the Bus-Pin Mapping API of the
ZKCM IRQ Control module. This enabled buses to be attached on demand and
removes the requirement for all bus-pin mapping information to be
detected all at once at boot.

A major sanity change to the MM is that Memory Streams no longer have
pointers to member functions, but the alloc() and free() functions now
directly reference code. There has also been a major overhaul in the
Orientation sequence, including a split into two threads. Several
integral classes have received the ability to be pre-allocated, which is
one change that has cleaned the kernel up /significantly/.

NUMA structure has been changed; to wit, there is no longer explicit
Memory locality, but the kernel now keeps memory latency information on
each CPU bank to use in deciding the best memory banks for a process.

Added IRQ processing to the kernel at a basic level, and an API
extension to ZKCM IRQ Control mod that enables the correct IRQ to be
processed when the kernel is entered in IRQ context.

v0.00.009

Toggle v0.00.009's commit message
I didn't get as much as I wanted done between v0.00.008 and v0.00.009…

…, but

the changes are significant enough, nonetheless. TBH I'll probably skip many
of them unintentionally, given that I won't remember a lot of them...

* Fixed a not-negligable number of bugs.
* Refined one or two classes.
* Finalized IBM-PC/x86-32 SMP support.
* Greatly refined the Interrupt Tributary.
* Expanded and re-fleshed the IRQ Control chipset-side API, creating a
  discrete ZKCM module for it.
* Very cleanly managed to hash out a method of handling the fact that
  IRQ pin routing layouts may change at runtime on a chipset.
* HardwareIDListC: Added the ability for it to far more memory efficient.
* WalkerPageRanger: Added "CreateMappingTo()", a generic mapping API call.
* Created the ZKCM Core chipset module which is a single chipset state
  tracker. This greatly simplified and sanitized the design of the whole
  of the IBM-PC chipset subdirectory.
* Added "Event Notification" to the IRQ Control mod, allowing the kernel
  to neatly inform the chipset of events of interest to it.
* Kernel will refuse to power on non-BSP CPUs if usingChipsetSmpMode()
  returns flase.
* Kernel will unconditionally immediately switch the chipset to SMP mode
  after a successful call to checkSmpSanity().
* Added checksum value calculation to the ACPI and MP table code.
* The kernel now handles pin based IRQs differently from MSI IRQs.

Next version increment should be a bit down the road. I hope to have the
scheduler completed, and a full understanding of where and how it would be
safe to begin introducing sleeplocks into the kernel.

v0.00.008

Toggle v0.00.008's commit message
One of the most significant version increments so far. The changes and

progress recorded with this release are innumerable and invaluable. They range
from bug fixes to new features both small and kernel-changing, including
amazing things like (actual) SMP support, a skeleton VFS, executable parser
array, and more.

An inexhaustive list of the new features included in this kernel version:

* Fully augmented the Memory Management subsystem to send and handle IPIs for
  TLB shootdown.
* Fully added support for Symmetric Multiprocessing on x86-32.
* Began work on the HVFS.
* Began work on the Executable parser array.
* Wrote a lot of the code associated with process scheduling.
* Wrote a lot of the code having to do with process spawning.
* Fixed a lot of critical bugs.
* Added support for the x86 LAPIC and I-O APIC chipset features.
* Added support for x86 CPU identification, although it almost always gets
  the CPU name wrong.
* Began work on the kernel's native driver subsystem which will be used for
  highly performance sensitive devices such as timers and possibly IRQ
  controllers.
* Major cleanups and C++-izations on many portions of the source code.

The next version increment should boast me having added a full scheduler and
having cleaned up the HVFS in preparation for the implementation of Zambezii's
Uniform Driver Interface environment.

-Shalom.

v0.00.007

Toggle v0.00.007's commit message
For now I've finished off NUMA Tributary's initialize2() sequence. Th…

…is means

that physical memory detection and setup is done.

v0.00.006

Toggle v0.00.006's commit message
Removed slam caching from the heap, added caching to numaMemoryBankC,

cleaned up constructors for most kernel classes, and LOTS of fixes everywhere.

I think I can confidently say that the kernel is more stable now than it's
every been.

Tagging before really moving on.

v0.00.005

Toggle v0.00.005's commit message
Nothin much for this version increment, but I've re-written memoryStr…

…eamC's

real_memAlloc(), and whatnot, and they're more versatile, and can take flags
like MEMALLOC_NO_FAKEMAP, MEMALLOC_PURE_VIRTUAL, etc. And IIRC, this version
introduces the heap, as well as some major refactoring of the PMM code for
NUMA.

I'm going to move on to Pmem enum, so I tagged before going on.

v0.00.004

Toggle v0.00.004's commit message
Tagging version 0.00.004, with the following features:

^ Added a Timer Tributary:
   * Supports a watchdog device.
   * Supports a bignum clock_t for keeping tick count.

^ Added an Interrupt Tributary:
   * A lot of it is stubbed, but:
   * Supports shared and non-shared interrupt vectors.
   * Has a sensible way of managing IRQs, even for unknown IRQs.

^ Refined Walker Page Ranger:
   * Eradicated a bug caused by bad inline ASM for INVLPG; Cause stale TLB
     entries to corrupt mappings in address spaces.
   * Fixed an "off by one" mistake in remapInc() and remapNoInc().
   * Pretty well tested.

^ Added a page fault handler on x86-32:
   * Supports demand paging.

^ Refined the Memory Tributary:
   * Re-wrote memAlloc and memFree.
   * Added a MEMALLOC_NO_FAKEMAP flag to memAlloc() to allow for non-demand
     paged allocations which must happen all at once.

^ Added a preliminary heap:
   * Uses James Molloy's SLAM algorithm for objects of
     size < PAGING_BASE_SIZE.

^ Added more bulk to chipset/ibmPc:
   * Defined a driver module for the interrupt controller (8259A).
   * Wrote a basic debug rivulet for use with the Debug Pipe.

^ Added a set of module APIs in /core/include/kernel/common/moduleApis:
   * API for Interrupt Controllers.
   * API for watchdog devices.

^ Added a Firmware Tributary:
   * Forms an abstraction interface for hding the details of different
     chipset/firmware combinations. Kernel just asks the firmware Tributary
     for information and gets it.
   * Provides rivulets for debug output, memory detection, NUMA detection,
     and several other firmware services.
   * Began porting x86Emu. As of now, it just builds. Haven't even tried
     to use it yet.

^ Added a multiplexing Debug Pipe class.
   * Can multiplex kernel debug messages into up to 4 devices.
   * Device multiplexing is transparent to the debug pipe.
   * Chipsets can define their own debug output devices, RS-232, terminal,
     etc, as they see fit.

GOALS FOR v0.00.005:
   * Clean up Memory Tributary.
   * Finish up and thoroughly test Pool Allocator.
   * Clean up NUMA Tributary.
   * Finish up Interrupt Tributary.
   * Re-do locks for x86-32 as per-architecture hand-optimized ASM.
   * Thoroughly test kernel.