Skip navigation.
Just your average bearded geek

Hacking the Linux kernel with KDevelop

KDevelop is really an awesome IDE. I've been doing a lot of IDE-jumping, being from the old school of Emacs and Vi (I know that sounds like a schizophrenic to mention both here), but not satisfied by the code-analysis abilities of these respectable ancestors and the lack of integration of code-browing tools. Sure, Emacs has ECB, but when I finally managed to configure the beast to work correctly on a reasonnably-sized project just to realize it is way too slow to be usable, I just dropped the whole Emacs thing in despair. There is of course CTags and CScope, which work rather well, but you have to manually update your databases and once again you hit a lack of integration and visual aid. What, Eclipse is capable to do flawless code referencing and error detection on Java since 10 years, and we do not have the same capability for a good C/C++ editor? Wait, we do - there is Qt Creator and Eclipse's CDT, so I shall dismiss them first before going on.

Qt Creator is actually pretty good an IDE. It's fast, relatively lightweight, has good code browsing capabilities and even Vi bindings (that's the one thing I don't want to lose from the old days - no Vi bindings, no code). I'd dare say it's close to perfection if you work with Qt only. Unfortunately for me, Qt is only part of my coding life. I have not been able to get it to work satisfyingly for the kernel - too Qt centric. For Qt, awesome (although KDevelop is at least as good IMHO). But for the kernel? Forget it.

CDT tries to bring the same level of integration for C/C++ to Eclipse as the JDT does. Well, this could be quite good too - and some people actually seem to use it to work on the kernel, but unfortunately after trying the same procedure and seeing that I have been using 1.5Gb of my memory to see red underlining all over the code, I was quite disappointed. Not to mention that using Eclipse on my quad-core, 8Gb dev beast still feels like driving a truck.

Meet KDevelop. Wait, hasn't this thing been in development for something like 10 years too? If you tried it in the KDE2/3 days, you may have been disappointed - I surely have been. Well, it may have taken time to come there, but let me tell you that as of today KDevelop is seriously underrated. It's by no way perfect (yet), but it's definitely the IDE that is the closest to my wishes:

  • Of course, it handles KDE and Qt code gracefully, as well as regular C/C++ projects. All my user-space needs covered.
  • It has Vi key bindings! No one considers doing serious coding without basic Vi bindings, does he?
  • Pretty good Git integration, including a visual 'git blame' that I love to use.
  • The project files format is simple and they can easily be edited to twiddle your settings without depending on the GUI and its limitations.
  • It does a rather good job at handling kernel code. There are still limitations - see below - but hopefully we will address them soon.

What is really nice about KDevelop with the kernel is that despite the huge quantity of code, navigation is fast and smooth (despite the memory usage that remains quite high. Good excuse to buy these 8GB of RAM). The Quick Open bar lets you instantly jump to a file, structure definition or function. The parser is not dumb like most of those I have seen (hi, CTags!) and does not rely on simple text matching to find where a variable is used - instead it really parses the code and knows what it means. Leaving the mouse on a label gives you all the information you possibly want to know about it, including the places where it is accessed. This is invaluable when exploring new code, and KDevelop does the best job at it. There is also a huge potential for code analysis, and I am quite confident we will also see things like function pointers resolving later (as the kernel is full of it, that would be a killer).

So, no more waiting - the next section will explain how to set up KDevelop to hack your kernel, and then we will review the limitations and shortcomings, which will hopefully be addressed in future versions.

Setting up a kernel project

We assume that you already git cloned the kernel source. Start KDevelop (version 4.2 or later).

The first thing to do is to go to Settings -> Configure KDevelop, then Background Parser and disable it. We will reenable it later, but you don't want it to go through Linux's 15 millions lines of code at this point.

Create the project using Project -> Open/Import Project. In the file selection dialog, go to the root of the Linux source and do not select any file. Click Next.

Put the name you like, and select the Generic Project Manager. Click Finish.

After a few seconds you should see your project appearing in the Projects panel. You can already open source files, but with the parser disabled it will not be much more useful than your averate text editor.

The next thing we want now is to re-enable the code parser so that we can navigate through the code. But since Linux code base is so big, we also want to limit it to the part that is useful for us. Also, we need to provide it with the right include paths.

Right-click the project in the Projects pane and select Open Configuration. There you can provide patterns for files to include and exclude from the project. Remove the * entry in the includes, and add the patterns corresponding to the parts of the kernel you are interested in. Most likely, you will at least want /kernel/*, /lib/* and /include/* since they cover the basics. Having /Documentation/* is also a good idea since it will allow you to lookup for documentation files through the Quick Open combo. What you want to be more selective about are the arch/ and drivers/ directories. Only include the arch subdirectory that corresponds to the architecture you plan to compile the kernel for. All the same, since driver code is so huge, you want to only include the sub-directories of drivers/ that you are interested in. The following screenshot gives you a good starting set of directories for working on ARM.

  

After clicking Ok, reenable the background parser. It will be working for a while, but should still be bearable (don't forget to set the number of jobs to the number of cores in your machine). You can now open a C file - however you will notice that most includes are not resolved and underlined with red. This is because KDevelop by default looks for include files in /usr/include, which is not really suitable for kernel work.

To fix this, you need to give it a list of include directories. Leave your mouse cursor on one of these unresolved includes, and click the Solve button, then Add custom include path. The following dialog can be a little confusing, so let's first explain how KDevelop handles custom include paths. Starting from the directory containing the file, and going up to the root directory, KDevelop searches for files names .kdev_include_paths that contains lists of include directories. The format is very simple: one line, one directory, given by its absolute path. This dialog just lets you create these files. The storage directory is where the .kdev_include_paths file will be created. Since we want it to apply to all the kernel, specify the root of your kernel source. Skip the Automatic resolution and just add absolute paths to the include/, arch/yourarch/include, and arch/yourarch/mach-yourmach/include, as shown by the following screen:

    

Alternatively, you can also add these paths manually into a file named .kdev_include_paths. This may actually be simpler.

Hit Ok (or hit F5 to reload your open file if you edited manually) and watch the background parser resolving your include files correctly. You can now use all the nice coding and code-browsing features of KDevelop on the kernel. Happy hacking!

  

Shortcomings

There are a small list of ennoying things at the moment. I hope to fix some (or better, all) of them in the future.

Frequently need to run “make mrproper”

This is an interesting behavior, and I don’t really know how to fix this. Once in a while, when you try to compile the kernel outside of the source tree, you will be asked to run “make mrproper”. The reason for that is that KDevelop seems to create an include/config directory inside the kernel sources. Running the command indeed cleans that directory (manually deleting it also works) and allows you to continue. Question is, why does this directory get created in the first place? There is one inside the build directory, but for some reason KDevelop seems to want to replicate it into the sources. Weird.

Kernel configuration include file not parsed

Once you have configured your kernels, a C include file is created in include/generated/autoconf.h that contains the zillions of macro definitions that allow you to enable/disable some features. These macros are extremely important for good code parsing. The file is automatically included at the beginning of every C file by passing the -include option to gcc, so it is not explicitly included from the code. Because of this, KDevelop cannot see these macros definitions and acts as if they did not exist - which is sometimes misleading. For instance, it will ignore all runtime PM code since it is only processed if the corresponding macro is defined.

Fixing this should only require minor twiddling of the C parser, and an option to automatically process some header files, similar to the include directories feature. It could even be saved in the same file to limit the automatic include to a subset of the project. Definitely the next on my tohack list.

__KERNEL__ not defined

All the same, many include files are shared between kernel and user space, and kernel-only parts are processed if the __KERNEL__ macro is defined. This macro is defined via the -D option of GCC, so once again KDevelop misses it. However, it should not be very difficult to add a macro definitions panel in the project configuration, like almost every IDE possesses.

C99 Initializations

This is  one of the rare missing things in the otherwise great KDevelop C parser. The kernel is full of C99 initializations that look like this:

static struct zorro_driver cirrusfb_zorro_driver = {
   .name        = "cirrusfb",
   .id_table    = cirrusfb_zorro_table,
   .probe        = cirrusfb_zorro_register,
   .remove        = __devexit_p(cirrusfb_zorro_unregister),
};  

KDevelop raises a parse error when it meets these, and of course the members are not available for code browsing and reference. Here the parser needs to be updated to support this form of initializer.

Conclusion

If you don't mind the initial project configuration and running into a few rough edges once in a while, kernel hacking with KDevelop is definitely something doable, and I'd even daresay enjoyable. As far as I'm concerned it is what gives me the most comfort. For beginners, the code browsing, online macro expanding, online documentation and quick open are invaluable features that largely counterbalance the few lacks. Hopefully I will find time to address them, but the best thing to do would be to have a kernel project type within KDevelop that could also be used to configure and compile the kernel and would take care of all this transparently. If people are willing to get involved, please feel free to leave a comment!

Let's fix it all :)

Hey there,

thanks for the kind words. I'll gladly help out in improving the situation even more. Here are some ideas if you want to investigate some stuff on your own:

a) make mrporper: Probably make is run in the wrong (i.e. the source) folder, instead of inside the build dir. Really needs to be investigated
b) include file: Maybe we can integrate that check into the custommake project manager and then pass that through to the cpp lang plugin
c) a proper macro/env definition UI is really missing, similar to how the "custom make file" manager already has. we just need something like that for custommake. And of course we should parse the makefiles properly to get these -D options and pass them along - the API for that exists, probably it's not implemented.
d) sadly, KDev's parser is actually a C++ parser. And afaik this C99 init is not part of C++ - not even C++11. But we can/should add it to the parser imo. there is a bug report about that.

What I'd like to hear from you (maybe on the ML) is some feedback on the performance of KDevelop. You mention mem consumption - I'd like to know where the mem is used exactly (massif?). Also I'd be interested if you notice some slow/sluggish things in the UI. Maybe you could even run KDevelop (git master!) through VTune with the kernel and find some more bottle necks. See: http://milianw.de/blog/vtune-and-kde

bye

Let's fix this indeed!

Hi Milian,

Thanks for your reply, your hard work on KDevelop and for integrating my patches. ;)

We should continue this discussion on the ML, but here are short quick answers anyway:

a) Isn't the generic project manager independant of make? Why would it need to run it in this case?

b) I thought it would make sense to have the automatically included files into .kdev_include_paths since it would allow to restrict the inclusion to some parts of a project. For the kernel project-wide settings are fine, but as far as the generic project manager is concerned we may as well try to be, well, generic. :p It seems, looking at the code, that these files can already hold other stuff (did not figure out what yet though), but if you think project wide settings are better, I'm fine with that too.

c) I wonder how far could Makefiles parsing could go for guessing -D parameters (especially if some variables need to be expanded). If this works, the same thing could be done for -include too. But letting the user specify these manually (along with a "guess from Makefile" button seems safer.

d) Yep, although a C++ parser, KDevelop's parser (unsurprisingly) does a great job with C. Here I guess being liberal is ok, as long as it "just works". I will try to get this done after I understand the parsing/DUchain stuff.

I will try to get the massif metrics you asked. As for memory consumption, parsing the parts of the code I have specified + a couple of drivers results in around 1GB of memory used, with a good dozen of files opened. By today standards, this seems very acceptable and much better than e.g. Eclipse anyway. Of course, if you want to parse the whole kernel tree, this is another story. The parser would take too much time anyway to make this practical, but this is not a real concern.

All in all KDevelop looks pretty fun to work on. In my student days I did quite a lot of static code analysis, so maybe I could also give a hand there, if time allows (which I unfortunately doubt). But first I'd like to fix these few things - after all, KDevelop is my main work tool, so I'd better shape it right for me. ;)

Awesome Post

Really good overview of the IDEs available. I've also used them all and pretty much share every comment on them. Nice insight on working with KDevelop of course.

Please tell me there's more

Please tell me there's more to come from you regarding KDevelop :-) I think it's really, really powerful. But more maturing from a new contributor would be really good to.

I have a couple of patches in

I have a couple of patches in progress - but my time for working on this is quite limited, so it takes more time than I'd like. But there are a couple of features I definitely wants to see in sometime soon.

Does anyone know if there is

Does anyone know if there is a way to add a custom define (such as __KERNEL__) to the parser?

Not easily, unless you use

Not easily, unless you use the custom builder. But I have started making a kernel plugin for KDevelop which does the job:

https://github.com/Gnurou/kdev-kernel

Only caveat is that it will only compile against KDevelop's current git (i.e. not against currently released versions). But if that does not scare you, you might want to give it a try.

Try

Try http://seascope.googlecode.com It understands c99 initializations

Thank you very much, My Kdev

Thank you very much, My Kdev is all set for some kernel hacking! that was really helpful! looking forward to seeing new content from you :)