Wednesday, August 19, 2009

Vnode cache design

I found a development quest in Jari OS: all begins from device drivers services design, some drivers are actually is a library that linking to service that support some hardware part, in this case driver developer folks need to have a dlopen()-family functions to link this libraries at runtime, for example if block service find a specific host it should be linked runtime, not statically. Well, OS hasn't dlopen-family support functions, saying more - dynamically linked ELF support are very early (it works, but it works slowly and takes many service time slice) - the reason is a poor file mapping support, you can map file, but when you will close file descriptor you will got a segfault on page fault in this mapped area. This is a complex problem, to support dlopen() , ELF support should be improved i.e. work with libraries and binaries should be going outside of process manager service and, not be done via read/write calls - one of the benefits will be a dlopen() support.
I know, that one of the weak place of the file subsystem layer is a libv2(it's a VFS-like layer, but implemented as a library that linked to each file system(don't confuse - VFS service is a filesystem manager, resource storage, redirection mechanism and so on)). Libv2 supports many POSIX features already, and operates via abstract vnode_t (in linux it's inode called) and there are too poor vnode cache, so every mapping is connected to vnode and if cache decide to destroy it - mapping can be lost. Like a solution is a map count handling, but it's a very simple - more complex solution is a creating more sophiscated cache.
Assume a file system with a high load, you are always performs lookups, maps, read, write operations - there are possible situation when vnode might not have positive map and open counters for a short amount of time, and there are possible rarely used vnode i.e. some vnode may be mapped sometime, but not accessible for a day - and it will waste you data structure and memory - and you cannot delete it at all. On other side - you cannot use vnode metadata that connected to real file system node, because lookup calls don't modify accessed time of this. Well, what the solution could be - first I determined several vnode stages with it's own features, second - I designed a table for vnodes that mapped, but doesn't accessed - it will not take resources as much as regular vnode takes, but virtually will present.
The next vnode stages might be:
  • strong vnode: not a candidate to move to the virtual vnode list, has a positive open and mapped count, and was recently used
  • potential oldies vnodes: has only mapped count or opened count positive and wasn't recently used
  • oldies vnode: vnodes in the virtual vnodes list
  • died vnodes: vnodes that was removed via unlink call
At defined period some thread will lookup the cache and decide what to do with vnode, this rate will be calculated by the file system activity (calls per last period). But what about died vnodes? Assume that you mapped file, somebody remove it, vnode id was taken by root only file ... well, died vnodes list as a low cost list that storage vnode of this types and when you will try to read or write to it, or you will unmap it, or got a page fault - information will be updated - you will got an error (or seg fault), file system will decrement the counters, and when this counters will zeroed - died vnode will go to heaven, and free it's id ;)
I'm not sure that I will implement all features at near time, but anyway this functionality will be added to libv2 - and I will continue my quest - I will improve page cache, improve memory events protocol, design and implement linker with dlopen ;)

Tuesday, July 07, 2009

My new kayak - seayak

On june I've bought a new kayak, manufacturer wrote in his list of products that this is a seayak type.
Well, like my double seat kayak it's a folding boat. I like folding kayaks in case of their mobility and storing place.
At current moment I've already made a two tests - is a two cycles of build, and yes - it's a on-water test.
Firstly I will give a more information about seayak:
Length: 4.9 meters
Weight: 22 kg
Frames: 4
Shipped with steering.
First built was taken up to 3 hours! And it's typically for new folding kayaks from triton ltd.
On first contruction I've broken one of the details and I was need to make an ugly hack that I've showed on image below:
Manufacturer has changed this detail for free via guarantee.
All construction set are typically , excluding head profile - it looks like on seayaks.
Second construction goes well and takes up to 1 hour and 10 minutes (via kayak manual it should take 62 minutes).
Letters with name of kayak model was removed and I think about new, original name for this boat.
Like a good design it has on-top stuff for small things, gps and other useful things on water (pack for cigarets or tobacco pipe ;) )
Generally it's a good model - it has a good on water performance, not so hard to build (not on first construction process;) ), good profile (I've tested it on big open water with big waves).
Now I'm thinking about new name ... and some modification - I want to add new on top fixtures for other things.
Other photos you can look here and here.

Saturday, May 09, 2009

SDK tools for OS development

New ideas came to my head, all about operating system features and development.

On last friday I decide to make some modifications on autotools stuff in GNU sed and try to compile it with Jari OS libc headers and functions. Yep - it's works! This means that our libc is ready for most of cases, many of them working now. This event gave me some new ideas about development tools for Jari OS: firstly I will port GNU gcc to Jari OS (I mean, that gcc will handle our ld scripts and libraries right, without many keys and other things), secondly I will construct SDK template (already done, but not publicied yet).
Also, to have a better support and big community additional feature needed - IDL integrated to SDK. Well, it's mean - that you have set of the templates for each type of service i.e. block device driver library, char device driver, file system and so on ...
In most cases you shouldn't known about specific internal things of system libraries, also you shouldn't care about interface changes (headers renaming, new functions, new order of calls and so on). This feature will generate libraries specific code in compile time and link your specific implementation. I.e. what will be written one time - will live all time ;)
For this feature it's better to use scheme (for templates, configs and other stuff) and some core of this will be written traditionally on C.