Toro kernel

A dedicated kernel for multi-threading applications.

Friday, May 12, 2017

Continuous Integration in torokernel (travis)


Hi folks, this weekend I spent some time playing around Travis. This is a tool for continuous integration which is very well integrated into Github. Travis makes very easy to build and run tests after a push or before a pull request for a given branch. 
I started my experiments with a simple idea: make a test to check if a simple application in Toro Kernel is correctly executed. To do this, I choose a simple application named ToroHello.pas which just writes "Hello World" to the serial console. To check this application, I set up Travis to do two things:
1. To compile ToroHello example and generate the booting image ToroHello.img.
2. To run the image in Qemu and test if the example worked.  
I based on a previous work in https://github.com/nielsAD/travis-lazarus. So 1) was easy to do and I got a ToroHello.img compiled by using Travis very quickly (see Figure 1). 
The task 2) took me a bit more time. I set up to run qemu after the building and to output the serial port to a file. Then, I tested if the output contains 'Hello world". In such a case, the test passed. Following pictures show the output of the job:

Figure 1. Travis compiling ToroHello.pas


Figure 2. Travis running ToroHello.img in Qemu and outputting the serial
Figure 2 shows the output from the serial port. We can see Toro's initialization at line 3574.

Figure 3. Result of the building

To summarizes, in a very short time I was able to compile ToroHello and then produce a test to verify its correct behavior. This is a simple example but it allows to show how powerful might be CI for a kernel. So I am planing to use Travis in two different levels of testing. The first level is to verify the compilation of the examples after a push and/or pull request. The second level is to write specific tests to measure performance, leaks, wrong behaviors, exception faults, etc. This needs however a lot more experimentation.

Matias

Monday, May 08, 2017

Porting genext2fs to Windows 10

Last weekend I spent some time to find a tool to create ext2 images in Windows. I found a wonderful tool named genext2fs which allows to create an ext2 image and to populate it with a directory structure. I could not find a way to compile it in windows so I put hands on it and I got a windows version that seems working. You can find the patch at the end of this post. However, it is a horrible patch that only makes the thinks work. If anyone knows a better way to do it, please don't hesitate to tell me. I compiled without any problem in cygwin64. We have to be careful when we run it since, for a reason that I don't understand, the program tries to put all current directory into the image. To prevent this, I just run it from the directory that I want to put into the image. To summarize, I am now able to create a ext2 image and populate it with a directory structure. For example, I am experimenting by populating ToroFiles.img with the files that are used by the webserver in Toro. To do so, I am running the following command: 

../genext2fs.exe -v -d ../ToroFiles -b 8192 ../ToroFiles.img

This generates a 8MB image named ToroFiles.img that contains the files in the directory ToroFiles. I executed the command inside the directory ToroFiles. In the next days, I will commit the example TorowithFilesystem.pas that uses this tool to populate ToroFiles.img. Together with the example, I will upload the instruction to run it so stay tuned!

Matias Vara

--------------------------------------------------------------------------------------------------------
This is a patch for genext2fs-1.4.1/genext2fs.c
--------------------------------------------------------------------------------------------------------

761c761
< readsize = readlink(path, buf, bufsize); /* 1st try */
---
> // readsize = readlink(path, buf, bufsize); /* 1st try */
1458c1458
< if(st->st_mode & S_IRGRP)
---
> /* if(st->st_mode & S_IRGRP)
1475c1475
< mode |= FM_ISVTX;
---
> mode |= FM_ISVTX; */
1631c1631
< lstat(dent->d_name, &st);
---
> stat(dent->d_name, &st);
1645c1645
< case S_IFLNK:
---
> // case S_IFLNK:
1652c1652
< case S_IFSOCK:
---
> // case S_IFSOCK:
1669c1669
< if (!S_ISDIR(st.st_mode) && !S_ISLNK(st.st_mode) && st.st_nlink > 1) {
---
> if (!S_ISDIR(st.st_mode) /* && !S_ISLNK(st.st_mode) */ && st.st_nlink > 1) {
1682c1682
< nod = mknod_fs(fs, this_nod, name, mode|FM_IFCHR, uid, gid, major(st.st_rdev), minor(st.st_rdev), ctime, mtime);
---
> // nod = mknod_fs(fs, this_nod, name, mode|FM_IFCHR, uid, gid, major(st.st_rdev), minor(st.st_rdev), ctime, mtime);
1685c1685
< nod = mknod_fs(fs, this_nod, name, mode|FM_IFBLK, uid, gid, major(st.st_rdev), minor(st.st_rdev), ctime, mtime);
---
> // nod = mknod_fs(fs, this_nod, name, mode|FM_IFBLK, uid, gid, major(st.st_rdev), minor(st.st_rdev), ctime, mtime);
1691c1691
< case S_IFSOCK:
---
> /* case S_IFSOCK:
1698c1698
< break;
---
> break; */
2392,2395c2392,2395
< if((pdir = open(".", O_RDONLY)) < 0)
< perror_msg_and_die(".");
< if(chdir(dopt[i]) < 0)
< perror_msg_and_die(dopt[i]);
---
> //if((pdir = open(".", O_RDONLY)) < 0)
> // perror_msg_and_die(".");
> //if(chdir(dopt[i]) < 0)
> // perror_msg_and_die(dopt[i]);
2397,2400c2397,2400
< if(fchdir(pdir) < 0)
< perror_msg_and_die("fchdir");
< if(close(pdir) < 0)
< perror_msg_and_die("close");
---
> //if(fchdir(pdir) < 0)
> // perror_msg_and_die("fchdir");
> //if(close(pdir) < 0)
> // perror_msg_and_die("close");
2593d2592

2595d2593


Saturday, March 04, 2017

Example of Toro Virtual Filesystem

Hi folks! I have spent some time to write an example of the use of the toro virtual filesystem. After fixing a couple of bugs, I manage to get an example named TorowithFileSystem.pas. This example together with others is hosted in torokernel/tests. The example can be tested by following the instructions here. I also committed a small ext2 image named ToroFiles.img that contains the files to make it run. For the moment, I am providing the whole image. I hope to provide a tool to create ext2 images. The example is very simple, it opens a file name index.hml located in the ext2 image and then it listens on port 80. When a connection arrives, it sends the content of the file. Briefly speaking, it is a sort of webserver. The example is a slightly modification of the ToroHttp.pas example. The only difference is that this example read the content from a file. To illustrate, I highlighted the differences in code in read and green (see Figure 1).  

Figure1. Differences between ToroHttp.pas and TorowithFilesystem.pas
          
In red, we can see the initialization part in which the disk ata0 is dedicated to the core 0 (local cpu). By doing this, only core can access to this device. Once dedicated, we are able to mount the first partition, which is identified with the minor number 6, by using the ext2 drive. After this, we are able to open the file index.html and read it until EOF. Then, the content of this file is sent when a new connection arrives. Enjoy!

Matias

Saturday, February 18, 2017

Compiling and Running Tests by using Lazarus 1.6.2

Hi folks! I am spending some time working on Lazarus and making all tests examples to compile and run directly from Lazarus. For the moment, I am only working on Windows 10. So far I managed to:
1. Compile ToroHello.pas directly from the IDE thus avoiding to execute any make from command line:


2. Run on Qemu ToroHello.img: 


I illustrated by using ToroHello, however, this also applies to all the tests. The changes are not still in master, instead I am working on the branch https://github.com/MatiasVara/torokernel/tree/supportlazaruscompilation. Fell free to clone and try. This only needs to install Lazarus 1.6.2 and FreePascal 3.0.0.


Matias