Tags: alpha123/zambesii
Tags
Bumping the kernel to v0.00.013, citing the achievement of a major mi… …lestone: We have UDI drivers executing, channels working, calls in and out of drivers working, and the initialization sequence genericized and repeatable. Apart from that, this version increment sees the addition of many beneficial changes to the kernel. * Uniform Driver Interface: * Added several metalanguages to the kernel tree: ^ udi_gio ^ udi_physio ^ udi_mgmt ^ zbz_root. * Introduced libzbzcore and libzudi. * Several syscalls for UDI implemented, including udi_mei_call and the udi_channel_* functions. * Implemented UDI channels. ^ Bind channels and normal channels. * Implemented UDI regions. * UDI drivers now mostly work. * The kernel now supports multi-parent devices, due to the fact that I now properly understand how udi_enumerate_ack() provides a metalanguage for child devices that it enumerates (through the ops_idx argument). * Cleaned up the VFS base types, added PATH_MAXLEN macros. * Migrated from the ZMESSAGE infrastructure to a new, cleaner, centralized, and better designed grand-central message queueing system, ::MessageStream. * Renamed subsystem macros accordingly. * Renamed function macros accordingly. * New user-queues for posting messages between threads in the same process. ^ Accompanying API is the MessageStream::postMessage() method. * Introduces the Lock::sOperationDescriptor struct, which is used to remove the threat of lost wakeups. * New kernel "IPC" class, for inter-thread communication: ZAsyncStream. ^ Enables a process to listen for connections and nominate a thread to handle requests over each new connection request. ^ Also has an "accept all" mode which allows messages to be taken without a pre-established connection. Useful for kernel server threads. * Style changes: * Renamed all classes in the kernel from myClassC to MyClass. * Renamed all structs in the kernel from myStructS to sMyStruct. * Renamed __KNULL to NULL. * Renamed __kprintf to printf. * New types for asynchronous programming assistance. ^ ::Callback, template<> ::_Callback, ::__kCallback. ^ AsyncResponse. * Floodplainn: * The birth of Floodplainn itself. * Implemented the Floodplainn VFS. * Implemented device types: * fplainn::Driver. * fplainn::Device. * fplainn::DeviceInstance. * fplainn::DriverInstance. * Devices that share the same driver will exist in the same address space. * Can now spawn driver processes. * Root virtual bus layout decided. * Connect() and Send() APIs allow normal processes to directly connect to a running device instance. * Added an array of mapping entries that translate udi metalanguage names into device class names. * Birth of the ZUI Server: * Created the ZUDI repository. * Created the ZUI server. * Implemented the ZUI parser classes. * Implemented the ZUI Server's driver matching code, as well as driver metadata loading, etc: ^ detectDriver, loadDriver, spawnDriver, instantiateDevice. * ZUI UDIProps database is now an integral part of the kernel build process. * Birth of the ZUM Server: * This is essentially the Zambesii UDI Management Agent. ^ Makes calls into drivers' MA channel on behalf of the kernel. ^ Sends udi_channel_event_ind calls into drivers on behalf of the kernel. * Lib and utility: * Printf has a recursive format string parser (%r), and has been somewhat reworked to be more decent. * Most list classes now have Iterators. * New RAII classes for heap objects. * Fixed serious errors in our implementation of operators new[] and delete[] due to lack of understanding. * The kernel has a new heap! ^ This new heap should be able to support realloc(). ^ It is an aggressive, self-analyzing monster that detects bugs. ^ Has exposed, single-handedly a whole lot of bugs. ^ Supports guard pages. * We now have stack traces! * General: * Kernel can now sort of build in Clang++. * The kernel now no longer supports response message redirection of kernel API response messages. * Namespace ipc:: exports a series of mapping and IPC data transfer methods for kernel use elsewhere. * On x86-32, demand mapping will try a max of 4 times for pmem. * The kernel no longer splits up threads using the nasty, bizzare scheme that we had in place before. We now have a power thread per CPU, and each CPU has its own power stack. * We now have early debug output, and a new kernel boot sequence which is far more micro-kernel-esque. * Kernel is now built with -fno-omit-frame-pointer.
Version increment to v0.00.012, as process spawning is mostly complet… …e. Moving on to the completion of the storage VFS next after some cleanup. This version increment sees a good few interesting features introduced, including but not limited to: * Process spawning. * Per-CPU Threads. * processStreamC, containerProcessC, containedProcessC, distributaryProcessC are all introduced by this version increment. * Memory Tributary now accepts a NO_SLEEP flag. * Kernel now properly handles top-level page table entry changes and propagates them across processes as they are pulled by the scheduler. * Updated the notes for x86-32/IBM-PC's placement addresses. * Unified and cleaned up the way that the kernel process is initialized. * Kernel process now has ID 0x10000, and process ID 0 is reserved for per-cpu threads. * Processes now have command lines, arguments, and environment variables. * Implemented a uniform callback infrastructure for threads to pull messages off of. * The kernel now uses the freestanding headers of the compiler being used to build it. * The behaviour of the "ID" member of processes and threads is slightly different. * Added basic support for LLVM. LLVM can be used to build the kernel, but the output binary does not run correctly. * Removed __kequalizerListC from use within the kernel vswamps, replacing it with the new infrastructure in slamCacheC. * Updated kernel design docs. * Distributary Trib processes can be executed now. * Changed the kernel's NOTICE/WARNING/ERROR/FATAL prefix strings; now they are shorter. * Fixed virtual memory leaks in libx86MP and libACPI. * taskTribC::block() now can take a lockC* argument and unlock this object after modifying the target CPU's schedqueue. This change is part of the solution to lost-wakeups from waiting on a single queue. A solution for threads waiting on multiple queues is yet to be found. * The stack structure of CPUs is now different: CPUs now have a per-cpu thread stack and a schedstack instead of a single sleepstack.
Version increment to v0.00.010. The next version increment is far more ambitious: will be hopefully incrementing to version 0.01.000. This version's main feature debut is the Timer Tributary and the Timer Streams. They together provide the kernel's timer API. * Began adding support for Clang to the build system. * Kernel now officially supports only C++03. No C++11, etc. * Loading bus-pin mappings for a bus will also cause the bus-pin mapping code to program the IRQ controller pins for that bus. * Created a platform specific placementAddresses.txt log. * Kernel boots all AP CPUs again now. ^ It does this in parallel too, using asynchronous timer events. * Task Tributary: * Fixed bugs in saveContextAndCallPull() and loadContextAndJump(). * Implemented basic scheduling functions such as yield(), dormant(), wake(), block(), unblock(). * __kclasses: * Implemented singleWaiterQueueC, a simple wait queue. ^ Has support for non-blocking pop(). * IBM-PC: * i8254: A good few refinements. ^ Holds an internal singleWaiterQueueC object for IRQ events. * Timer Tributary: * Defined the structure and code for the event dequeuer thread. ^ Checks multiple queues, with a dedicated, prioritized message queue for control messages from the kernel. ^ Should be reasonably free of races. * Defined the prototypes for the timer API. * Defined the layout of the Timer Stream event objects. * Added support for allowing the chipset to hook a clock callback routine into a timer Queue to keep the system clock updated. Will probably be obsoleted soon enough. * Process/Thread: * Each thread now has its own event queue for each event. * Moved the pendingEvents BMP out of taskC into processStreamC. * Threads are no longer spawned with local IRQs disabled by default. * CPU Tributary: * InterCpuMessagerC: Got a moderate overhaul of its state machine. ^ No longer controls local IRQ acceptance for its parent CPU. * Added powerManagerC to each CPU. * Created an Object Oriented class for LAPICs on x86. * Interrupt Tributary: * Re-structured the way interrupts are permeated to the handling logic. * Now split into pinIrqMain(), exceptionMain(), swiMain() msiIrqMain(). * Pin IRQs now have a new ZKCM ISR exit status: ZKCM_ISR_SUCCESS_AND_RETIRE_ME, which removes the ISR after executing it. Fixes a race condition. * x86 (-32): * Implemented basic support for the Debug Registers.
Added a license to the kernel tree. The kernel is now officially GPLv2 licensed. Also added a readme with information on the commercial intent for the kernel. This version increment also introduces changes to the build system that include a smooth, ./configure && make style build, with an interactive configure that can be extended to support multiple build toolchains. A lot of the work done in this version increment has to do with the kernel's time types, the timer subsystem, and scheduling since this was the core goal of this next version. Specific notable time types implemented are timeS, date_t and timestampS. The kernel takes time to be gregorian, and userspace libraries are used to convert into POSIX epocs. The kernel now also takes note of the date and time it was booted (boot timestamp). The ZKCM Driver subsystem and APIs have been revamped to be object oriented, extensible, and to be somewhat uniform with the UDI model, enabling them to co-exist synergetically with the soon-to-be-implemented UDI environment. Various minor bugs were fixed, and I also added NULL pointer checking to the functions in the C string lib. Some language level improvements have also been made: almost all HLL files in the kernel are now C++ and not C. Additionally, I began "C++izing" the kernel a bit. Chipset related improvements include a more sane chipset SMP mode switch, better detection of bad SMP boards, and the implementation of a very important chipset specific API, the Bus-Pin Mapping API of the ZKCM IRQ Control module. This enabled buses to be attached on demand and removes the requirement for all bus-pin mapping information to be detected all at once at boot. A major sanity change to the MM is that Memory Streams no longer have pointers to member functions, but the alloc() and free() functions now directly reference code. There has also been a major overhaul in the Orientation sequence, including a split into two threads. Several integral classes have received the ability to be pre-allocated, which is one change that has cleaned the kernel up /significantly/. NUMA structure has been changed; to wit, there is no longer explicit Memory locality, but the kernel now keeps memory latency information on each CPU bank to use in deciding the best memory banks for a process. Added IRQ processing to the kernel at a basic level, and an API extension to ZKCM IRQ Control mod that enables the correct IRQ to be processed when the kernel is entered in IRQ context.
I didn't get as much as I wanted done between v0.00.008 and v0.00.009… …, but the changes are significant enough, nonetheless. TBH I'll probably skip many of them unintentionally, given that I won't remember a lot of them... * Fixed a not-negligable number of bugs. * Refined one or two classes. * Finalized IBM-PC/x86-32 SMP support. * Greatly refined the Interrupt Tributary. * Expanded and re-fleshed the IRQ Control chipset-side API, creating a discrete ZKCM module for it. * Very cleanly managed to hash out a method of handling the fact that IRQ pin routing layouts may change at runtime on a chipset. * HardwareIDListC: Added the ability for it to far more memory efficient. * WalkerPageRanger: Added "CreateMappingTo()", a generic mapping API call. * Created the ZKCM Core chipset module which is a single chipset state tracker. This greatly simplified and sanitized the design of the whole of the IBM-PC chipset subdirectory. * Added "Event Notification" to the IRQ Control mod, allowing the kernel to neatly inform the chipset of events of interest to it. * Kernel will refuse to power on non-BSP CPUs if usingChipsetSmpMode() returns flase. * Kernel will unconditionally immediately switch the chipset to SMP mode after a successful call to checkSmpSanity(). * Added checksum value calculation to the ACPI and MP table code. * The kernel now handles pin based IRQs differently from MSI IRQs. Next version increment should be a bit down the road. I hope to have the scheduler completed, and a full understanding of where and how it would be safe to begin introducing sleeplocks into the kernel.
One of the most significant version increments so far. The changes and progress recorded with this release are innumerable and invaluable. They range from bug fixes to new features both small and kernel-changing, including amazing things like (actual) SMP support, a skeleton VFS, executable parser array, and more. An inexhaustive list of the new features included in this kernel version: * Fully augmented the Memory Management subsystem to send and handle IPIs for TLB shootdown. * Fully added support for Symmetric Multiprocessing on x86-32. * Began work on the HVFS. * Began work on the Executable parser array. * Wrote a lot of the code associated with process scheduling. * Wrote a lot of the code having to do with process spawning. * Fixed a lot of critical bugs. * Added support for the x86 LAPIC and I-O APIC chipset features. * Added support for x86 CPU identification, although it almost always gets the CPU name wrong. * Began work on the kernel's native driver subsystem which will be used for highly performance sensitive devices such as timers and possibly IRQ controllers. * Major cleanups and C++-izations on many portions of the source code. The next version increment should boast me having added a full scheduler and having cleaned up the HVFS in preparation for the implementation of Zambezii's Uniform Driver Interface environment. -Shalom.
Removed slam caching from the heap, added caching to numaMemoryBankC, cleaned up constructors for most kernel classes, and LOTS of fixes everywhere. I think I can confidently say that the kernel is more stable now than it's every been. Tagging before really moving on.
Nothin much for this version increment, but I've re-written memoryStr… …eamC's real_memAlloc(), and whatnot, and they're more versatile, and can take flags like MEMALLOC_NO_FAKEMAP, MEMALLOC_PURE_VIRTUAL, etc. And IIRC, this version introduces the heap, as well as some major refactoring of the PMM code for NUMA. I'm going to move on to Pmem enum, so I tagged before going on.
Tagging version 0.00.004, with the following features: ^ Added a Timer Tributary: * Supports a watchdog device. * Supports a bignum clock_t for keeping tick count. ^ Added an Interrupt Tributary: * A lot of it is stubbed, but: * Supports shared and non-shared interrupt vectors. * Has a sensible way of managing IRQs, even for unknown IRQs. ^ Refined Walker Page Ranger: * Eradicated a bug caused by bad inline ASM for INVLPG; Cause stale TLB entries to corrupt mappings in address spaces. * Fixed an "off by one" mistake in remapInc() and remapNoInc(). * Pretty well tested. ^ Added a page fault handler on x86-32: * Supports demand paging. ^ Refined the Memory Tributary: * Re-wrote memAlloc and memFree. * Added a MEMALLOC_NO_FAKEMAP flag to memAlloc() to allow for non-demand paged allocations which must happen all at once. ^ Added a preliminary heap: * Uses James Molloy's SLAM algorithm for objects of size < PAGING_BASE_SIZE. ^ Added more bulk to chipset/ibmPc: * Defined a driver module for the interrupt controller (8259A). * Wrote a basic debug rivulet for use with the Debug Pipe. ^ Added a set of module APIs in /core/include/kernel/common/moduleApis: * API for Interrupt Controllers. * API for watchdog devices. ^ Added a Firmware Tributary: * Forms an abstraction interface for hding the details of different chipset/firmware combinations. Kernel just asks the firmware Tributary for information and gets it. * Provides rivulets for debug output, memory detection, NUMA detection, and several other firmware services. * Began porting x86Emu. As of now, it just builds. Haven't even tried to use it yet. ^ Added a multiplexing Debug Pipe class. * Can multiplex kernel debug messages into up to 4 devices. * Device multiplexing is transparent to the debug pipe. * Chipsets can define their own debug output devices, RS-232, terminal, etc, as they see fit. GOALS FOR v0.00.005: * Clean up Memory Tributary. * Finish up and thoroughly test Pool Allocator. * Clean up NUMA Tributary. * Finish up Interrupt Tributary. * Re-do locks for x86-32 as per-architecture hand-optimized ASM. * Thoroughly test kernel.
PreviousNext