Here's to hoping the copy/paste out of ChatZilla doesn't get screwed up in email :) cormander huzzah, I haven't been on IRC in years... kragniz cormander: \o cormander howdy kragniz cormander: you should spend more time here! cormander let's just say, I have a hard enough time managing distractions and social anxiety without being in a chat room all day long kragniz maybe I will learn something similar one day kragniz anyway, kpatch stuff! cormander pardon my ignorance - but do I mentor one student, or many? cormander and if you don't know the answer, that's fine kragniz just the one is what people say cormander well so far it's you and one other person that's contacted me directly cormander though someone who is closer to my timezone is probably better suited kragniz you'll need to pick one, I'm pretty sure - the actual selection takes place in a few weeks, though cormander okay so remind me - was kpatch your first choice? one of you said it wasn't your first choice ... kragniz you should talk with the centos gsoc admins, though cormander yeppers will do kragniz now you ask that question, I guess it is cormander so I have some recommended reading for you regarding kernel patching kragniz I was weighing it and tinykdump equally in my head previously cormander which is, in some ways, a shameless self promotion... kragniz fire away! cormander er, I would have sworn I posted about kpatch on my blog cormander okay plan B, my older post about ksplice kragniz heh cormander holy crap that was 4 years ago. I feel old now. cormander http://cormander.com/2011/08/how-to-use-the-ksplice-raw-utilities/ kragniz I'm guessing this one? http://cormander.com/2011/08/how-to-use-the-ksplice-raw-utilities/ kragniz hehe cormander not to be a copycat or anything, but that's roughly the same kind of process I'd like to work with cormander the kpatch utilities (as of about 6 months ago - haven't looked at them since) worked similar to it kragniz yup, they do cormander each tree needs to be treated like it's own line of code, because not all patches are applicable to the older ones, and not all fixes go into each version exactly the same way kragniz (as of last week) cormander and by tree, I mean version/release of the kernel cormander so one patch may go into 4 different versions exactly the same way, but there will come a point where it needs to be modified to work with a newer version cormander and vice versa cormander originally I wanted to maintain one big patch for each version and just update it each time a new version came out cormander but, as I started actually working on it, I discovered that until kpatch solves some technical problems with kernel functions that are always on the stack cormander the bigger the patch, the more risky it is that once you remove it, it becomes difficult to re-insert it again kragniz okay, that's interesting cormander we don't want to hit a patch with a heavy kernel thread and fight the battle of yanking it / re-probing it every time cormander because most patches don't run that risk cormander but that creates a different techincal problem - tracking which functions are being touched cormander not a hard problem to solve if you stay consistent, but then you run into issues with being incompatible with other kpatch efforts, should there ever be any, if everyone doesn't agree on an API for it kragniz couldn't that be solved by adding that to upstream kpatch-build? cormander ie; my project tpe-lkm modifies live kernel code, and does it outside of kpatch cormander https://github.com/cormander/tpe-lkm/ kragniz clicks cormander tpe-lkm is better at supporting a wide range of kernel versions, but it's not as fine-grained as kpatch cormander and thus, is limited in what kinds of patching it can do kragniz as an aside, that project is really cool! cormander thanks cormander now, we're not going to be able to solve compatibility issues with non-kpatch projects like that cormander but it's worth thinking about kragniz but getting something that keeps in mind other systems would be good? cormander yeah kragniz okay cormander as an aside, my primary motivation into learning kpatch was to apply portions of grsecurity via kpatch cormander just like how I apply some of them via tpe-lkm, cormander but kpatch can do many more of them kragniz interesting cormander the bump I hit was, which derailed my efforts, that problem with functions in heavy use cormander it's probably solvable, but let's just say I only have so much motivation to solve certain problems kragniz hehe kragniz have you played with the livepatching in linux 4.0? cormander no cormander I've barely scratched the surface with kpatch on EL7 kragniz (just changing the topic slightly) kragniz okay cormander I went hardcore into the kpatch code and submitted patches and all that to https://github.com/dynup/kpatch cormander for a few weeks cormander then life happened again and knocked me off the wagon kragniz yeah, I had a look at them kragniz life is a sad thing cormander have you read this? cormander https://github.com/dynup/kpatch#limitations kragniz yup cormander we need a strategy to handle patches that touch those limitations cormander which may just end up being "sorry folks, you actually have to reboot now" cormander but I'm leaving that as a last resort cormander for example, kpatch uses something called shodows cormander and then you can redirect the use of statically allocated data and use the shadows instead cormander *shadows kragniz shadows being new copies of the old data? cormander yes cormander one very important thing we need, moreso than the API to marry all our different patches together (so they don't stomp on each other) cormander is some kind of automated QA testing kragniz CI on each patch? cormander yes kragniz okay! cormander something that boots a VM, applies each patch in sequence, and runs some sort of kernel test suite to make sure it doesn't go kablooey cormander and then do it for each kernel version we support kragniz doing that in a VM is okay? cormander yes kragniz cool kragniz that sort of thing is fun kragniz although, setting that up may be a project in itself cormander the good news is, the "kernel test suite" would only need to be limited to calling the functions that are patched cormander so initially, the test suite is small, and we simply write more tests as patches touch more symbols cormander I agree, and I need to specifically talk to kbsingh about that kragniz yup, so you could collect the functions being patched an automatically build a test suite? cormander if you think that could be done, go for it cormander I was thinking along the lines of just small c code a.out executables that make certain syscalls kragniz that sounds good, also kragniz simple is better, most of the time cormander occam's razor cormander so, do you think this stuff is up your alley? kragniz totally! cormander full disclosure: I'm not as skilled with kernel C code as you might think, considering I've written quote a lot of kernel module code myself cormander I shoot myself in the foot more than I care to admit kragniz I warn you, though, I'm still learning kernel level stuff kragniz we can learn together! cormander great! we're both good enough to be dangerous kragniz I've done a fair amount of userland C, though python is my pet cormander perl here kragniz shakes his fist cormander alright so - what timezone are you in? I hope this isn't too late in the evening for you kragniz normally utc, but I'm in est at the moment cormander ahh! 5am utc cormander too early? kragniz I work on openstack, so I'm used to late night irc meetings! kragniz so, more about the general scope of the project: cormander I'll be as accommodating as I can - although I do get up at 4am from time to time myself kragniz should this be a system which is only designed to run on centos? anything rpm based? anything running linux? cormander good questions, cormander for now, only designed for EL7 cormander yes RPM based cormander needs to be compatible with yum cormander the API of what function(s) each of our kpatches use could be as simple as cormander making an rpm "virtual" dependancy cormander kernel-kpatch-function-X cormander where X is the functions it touches kragniz hmm, that would be interesting cormander I forget what I'm thinking of is called cormander where you specify this is the only package that can touch this cormander installing another with it will get rejected kragniz conflicts? cormander yes! cormander wow it's been a while kragniz hehe cormander there will be situations where two patches touch the same functions cormander they need to get rolled together into an update for that particular patch cormander so, occationally, an already released kpatch rpm will get an update cormander but mostly, it's just installing new kpatch rpms kragniz okay, so when that new package comes along, the other patches will be removed cormander updated cormander and by being updated, the old one will get rmmod'd, and re-insmod with the new one cormander or however kpatch handles that kragniz if it conflicts, shouldn't the whole conflicting package be uninstalled? cormander if it conflicts, and it's from our repository, that means we didn't do a good QA job cormander I'll explain it like this; cormander we have patches A B C D E F G cormander all touching different things cormander patch H comes out, and touches what A does cormander instead of releasing H as a new rpm cormander we update A to include H kragniz right, I get you now! cormander then, patch J comes out, it touches what D and G do kragniz I thought you meant bundle As patch with H and remove A cormander we update D to include G and J, and have it "depricate" G cormander or whatever term rpm uses for depricate ... where it uninstalls it upon install kragniz yup, that makes sense cormander and this, my friend, is where the QA matters most kragniz indeed! cormander like I said, if an rpm conflicts, we f***ed up kragniz yup cormander so, chew on that for now. I actually have to get up early tomorrow cormander see you tomorrow around this time? kragniz alrighty kragniz sounds good! kragniz thanks for the thoughts! cormander goes "away" - if there is such a thing in IRC kragniz goes to bed