gsoc March 2015

gsoc@lists.centos.org

25 participants
36 discussions

yum plugins for docker
by Jim Perrin 17 Mar '15

17 Mar '15

I've been asked a couple times privately for information about this, so here's my current thinking on the project goals. must-have: 1. be able to exec (or otherwise connect to a container) and perform a 'yum update' of the container, commiting the results upon successful completion. eg - yum dockerupdate -y centos # run yum -y update in the centos:latest container. 2. Traditional yum interaction is expected (prompting for 'y/n', or accepting options like -y) 3. Passing exit values from the container to the host. 4. Exit with message on unsupported containers. 4. documentation and invalid-command syntax fails to usage/help. nice-to-have: 1. tab complete support for commands as well as available containers. 2. support for non-rpm based updates (support apt for debian/ubuntu based containers) questionable: glob/regex support to perform update on every matching container Comments/Questions? -- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77

2 1

kpatch irc log 2015-03-16
by Corey Henderson 17 Mar '15

17 Mar '15

Here's to hoping the copy/paste out of ChatZilla doesn't get screwed up in email :) cormander huzzah, I haven't been on IRC in years... kragniz cormander: \o cormander howdy kragniz cormander: you should spend more time here! cormander let's just say, I have a hard enough time managing distractions and social anxiety without being in a chat room all day long kragniz maybe I will learn something similar one day kragniz anyway, kpatch stuff! cormander pardon my ignorance - but do I mentor one student, or many? cormander and if you don't know the answer, that's fine kragniz just the one is what people say cormander well so far it's you and one other person that's contacted me directly cormander though someone who is closer to my timezone is probably better suited kragniz you'll need to pick one, I'm pretty sure - the actual selection takes place in a few weeks, though cormander okay so remind me - was kpatch your first choice? one of you said it wasn't your first choice ... kragniz you should talk with the centos gsoc admins, though cormander yeppers will do kragniz now you ask that question, I guess it is cormander so I have some recommended reading for you regarding kernel patching kragniz I was weighing it and tinykdump equally in my head previously cormander which is, in some ways, a shameless self promotion... kragniz fire away! cormander er, I would have sworn I posted about kpatch on my blog cormander okay plan B, my older post about ksplice kragniz heh cormander holy crap that was 4 years ago. I feel old now. cormander http://cormander.com/2011/08/how-to-use-the-ksplice-raw-utilities/ kragniz I'm guessing this one? http://cormander.com/2011/08/how-to-use-the-ksplice-raw-utilities/ kragniz hehe cormander not to be a copycat or anything, but that's roughly the same kind of process I'd like to work with cormander the kpatch utilities (as of about 6 months ago - haven't looked at them since) worked similar to it kragniz yup, they do cormander each tree needs to be treated like it's own line of code, because not all patches are applicable to the older ones, and not all fixes go into each version exactly the same way kragniz (as of last week) cormander and by tree, I mean version/release of the kernel cormander so one patch may go into 4 different versions exactly the same way, but there will come a point where it needs to be modified to work with a newer version cormander and vice versa cormander originally I wanted to maintain one big patch for each version and just update it each time a new version came out cormander but, as I started actually working on it, I discovered that until kpatch solves some technical problems with kernel functions that are always on the stack cormander the bigger the patch, the more risky it is that once you remove it, it becomes difficult to re-insert it again kragniz okay, that's interesting cormander we don't want to hit a patch with a heavy kernel thread and fight the battle of yanking it / re-probing it every time cormander because most patches don't run that risk cormander but that creates a different techincal problem - tracking which functions are being touched cormander not a hard problem to solve if you stay consistent, but then you run into issues with being incompatible with other kpatch efforts, should there ever be any, if everyone doesn't agree on an API for it kragniz couldn't that be solved by adding that to upstream kpatch-build? cormander ie; my project tpe-lkm modifies live kernel code, and does it outside of kpatch cormander https://github.com/cormander/tpe-lkm/ kragniz clicks cormander tpe-lkm is better at supporting a wide range of kernel versions, but it's not as fine-grained as kpatch cormander and thus, is limited in what kinds of patching it can do kragniz as an aside, that project is really cool! cormander thanks cormander now, we're not going to be able to solve compatibility issues with non-kpatch projects like that cormander but it's worth thinking about kragniz but getting something that keeps in mind other systems would be good? cormander yeah kragniz okay cormander as an aside, my primary motivation into learning kpatch was to apply portions of grsecurity via kpatch cormander just like how I apply some of them via tpe-lkm, cormander but kpatch can do many more of them kragniz interesting cormander the bump I hit was, which derailed my efforts, that problem with functions in heavy use cormander it's probably solvable, but let's just say I only have so much motivation to solve certain problems kragniz hehe kragniz have you played with the livepatching in linux 4.0? cormander no cormander I've barely scratched the surface with kpatch on EL7 kragniz (just changing the topic slightly) kragniz okay cormander I went hardcore into the kpatch code and submitted patches and all that to https://github.com/dynup/kpatch cormander for a few weeks cormander then life happened again and knocked me off the wagon kragniz yeah, I had a look at them kragniz life is a sad thing cormander have you read this? cormander https://github.com/dynup/kpatch#limitations kragniz yup cormander we need a strategy to handle patches that touch those limitations cormander which may just end up being "sorry folks, you actually have to reboot now" cormander but I'm leaving that as a last resort cormander for example, kpatch uses something called shodows cormander and then you can redirect the use of statically allocated data and use the shadows instead cormander *shadows kragniz shadows being new copies of the old data? cormander yes cormander one very important thing we need, moreso than the API to marry all our different patches together (so they don't stomp on each other) cormander is some kind of automated QA testing kragniz CI on each patch? cormander yes kragniz okay! cormander something that boots a VM, applies each patch in sequence, and runs some sort of kernel test suite to make sure it doesn't go kablooey cormander and then do it for each kernel version we support kragniz doing that in a VM is okay? cormander yes kragniz cool kragniz that sort of thing is fun kragniz although, setting that up may be a project in itself cormander the good news is, the "kernel test suite" would only need to be limited to calling the functions that are patched cormander so initially, the test suite is small, and we simply write more tests as patches touch more symbols cormander I agree, and I need to specifically talk to kbsingh about that kragniz yup, so you could collect the functions being patched an automatically build a test suite? cormander if you think that could be done, go for it cormander I was thinking along the lines of just small c code a.out executables that make certain syscalls kragniz that sounds good, also kragniz simple is better, most of the time cormander occam's razor cormander so, do you think this stuff is up your alley? kragniz totally! cormander full disclosure: I'm not as skilled with kernel C code as you might think, considering I've written quote a lot of kernel module code myself cormander I shoot myself in the foot more than I care to admit kragniz I warn you, though, I'm still learning kernel level stuff kragniz we can learn together! cormander great! we're both good enough to be dangerous kragniz I've done a fair amount of userland C, though python is my pet cormander perl here kragniz shakes his fist cormander alright so - what timezone are you in? I hope this isn't too late in the evening for you kragniz normally utc, but I'm in est at the moment cormander ahh! 5am utc cormander too early? kragniz I work on openstack, so I'm used to late night irc meetings! kragniz so, more about the general scope of the project: cormander I'll be as accommodating as I can - although I do get up at 4am from time to time myself kragniz should this be a system which is only designed to run on centos? anything rpm based? anything running linux? cormander good questions, cormander for now, only designed for EL7 cormander yes RPM based cormander needs to be compatible with yum cormander the API of what function(s) each of our kpatches use could be as simple as cormander making an rpm "virtual" dependancy cormander kernel-kpatch-function-X cormander where X is the functions it touches kragniz hmm, that would be interesting cormander I forget what I'm thinking of is called cormander where you specify this is the only package that can touch this cormander installing another with it will get rejected kragniz conflicts? cormander yes! cormander wow it's been a while kragniz hehe cormander there will be situations where two patches touch the same functions cormander they need to get rolled together into an update for that particular patch cormander so, occationally, an already released kpatch rpm will get an update cormander but mostly, it's just installing new kpatch rpms kragniz okay, so when that new package comes along, the other patches will be removed cormander updated cormander and by being updated, the old one will get rmmod'd, and re-insmod with the new one cormander or however kpatch handles that kragniz if it conflicts, shouldn't the whole conflicting package be uninstalled? cormander if it conflicts, and it's from our repository, that means we didn't do a good QA job cormander I'll explain it like this; cormander we have patches A B C D E F G cormander all touching different things cormander patch H comes out, and touches what A does cormander instead of releasing H as a new rpm cormander we update A to include H kragniz right, I get you now! cormander then, patch J comes out, it touches what D and G do kragniz I thought you meant bundle As patch with H and remove A cormander we update D to include G and J, and have it "depricate" G cormander or whatever term rpm uses for depricate ... where it uninstalls it upon install kragniz yup, that makes sense cormander and this, my friend, is where the QA matters most kragniz indeed! cormander like I said, if an rpm conflicts, we f***ed up kragniz yup cormander so, chew on that for now. I actually have to get up early tomorrow cormander see you tomorrow around this time? kragniz alrighty kragniz sounds good! kragniz thanks for the thoughts! cormander goes "away" - if there is such a thing in IRC kragniz goes to bed

1 0

Corey's GSoC "office hours"
by Corey Henderson 17 Mar '15

17 Mar '15

Hi all, I'm mentoring the kpatch delivery for this year's GSoC. My apologies if I'm a little late to the party. My availability during my work day is limited to email from my mobile device. As such, I've decided to set a time each weekday evening that I can consistently be online and available for chat and other live conversation. 10pm to 11pm US Mountain time on weekdays I'll be online. I plan on putting in about 2 hours each night, so I'll be online before 10pm sometimes, and sometimes after 11pm based on my schedule - but from 10 to 11 I can certainly commit to. If you don't see me, send an email to this list or me directly with what medium you're online at, and I should see it pretty quick. Thanks, -- Corey

2 1

GSOC Proposal discussion for Extendable system hardening tool
by Aneesh Dogra 12 Mar '15

12 Mar '15

Hello, My name is Aneesh Dogra. I am studying computer science in IIIT Delhi. I have been involved in a few open source projects (primarly libav and sugarlabs) since 2-3 years. I have won GCI twice [1] [2] and mentored in GSoC for Sugarlabs (2 years ago) and been a GSOC student last year. I have a fairly active/updated github [3] and linkedin [4] profile. I have experience in security pentesting, C, python and shell. Extendable System hardening tool: Here's what I understand of the idea: Create a tool to enforce some secure habits on the server depending on the inputs (modes/recipes). The trick here is the make this very general. Here's my proposed solution, after thinking over it for a while: 1) There will be 2 components of the tool, recipe and modes. - A recipe will be provided through a cdn to users for different modes. - Each mode's recipe will have data so as to what to do to strengthen the system. 2) The tool would parse the recipe file and execute the instructions listed in the file, now either we could demand some bash scripts in the recipe file, but this won't be neat. I was thinking of making predefined functions in the tool and using some macros to represent them in the recipe file. I'll need some help on this. Can we discuss this? Is having bash code written in the recipes a bad idea? [1] : http://google-opensource.blogspot.in/2012/02/google-code-in-2011-grand-priz… [2] : http://google-opensource.blogspot.in/2013/02/google-code-in-2012-grand-priz… [3] : https://github.com/lionaneesh [4] : linkedin.com/in/aneeshdogra [5] : https://github.com/lionaneesh/jsonDB -- Regardless, I hope you're well and happy - Aneesh

2 1

GSOC 2015 Implement and create new documentation toolchain
by kunaal jain 12 Mar '15

12 Mar '15

Hello, I am interested in working on this idea. Can anyone please connect to the mentor, so I can discuss details, requirements and other stuff. Thanks, Kunaal Jain kunaalus(a)gmail.com

2 1

Extendable system hardening tool - Proposal
by vladiksonic＠mail.ru 12 Mar '15

12 Mar '15

Hello everyone! I wrote earlier Here is a link to my schedule: https://github.com/hotab/CentOS-application/blob/master/README.md I did not add personal info there just yet. I am planning to spend around 20-30 hours per week on this in summer. I might also start early (see previous message) Did I miss anything? Did I make correct time estimation there? I am open for criticism. Looking forward to hearing from you, Vladislav Babkin P.S. Adding the message I send earlier to make it simpler to find :) P.S.2 While I was writing the proposal, I also answered my questions 1 (sh scripts are too good) and 3 (it should work internally and allow external audit) > Good day there! > My name is Vladislav Babkin, I am from Ukraine, and I want to contribute by helping developing this tool > I have some questions: > 1) Aren't sh scripts good enough to audit a policy? Well, we will need some additional tooling, like a tool to try and open a connection to a port, etc. > 2) Should auditing tool be able to try and fix the problems? For example, if it connects to web-server as root - maybe it should connect to it and close that possibility (if the user adds a flag to do so)? > 2) When ensuring a policy, should this tool autoinstall software as well if is not present, or should it just fail? > 3) Should it work internally on the server, or can it be run somewhere on the outside with access to the machine through ssh or alike? > 4) I am right now working on a starup (and will do so in summber), and I will have my exams in June, so I probably will start working earlier, Hope that is not a problem :) > I'm looking forward to hearing from you, > Vladislav Babkin

1 0

Re: [CentOS-gsoc] centos-gsoc
by Lalatendu Mohanty 11 Mar '15

11 Mar '15

On 03/11/2015 12:47 AM, shuai wu wrote: > Dear Mr.Mohanty, > > I am shuai wu, a master student of university of florida. > > I am very interested in your GSOC idea—CentOS Cloud Storage components, which is exactly the field that me love. > > I have gained project experience of distributed system from class, such as building peer-to-peer networking, implement twitter-like engine.(those projects can be find at my github: https://github.com/westsnow) > > And I was so excited to find your idea on the CentOS-GSOC page. How cool it would be to put what I have learned to an open source project! > > Could you please give me some more detailed information about your project? > > Thanks very much and I am looking forward to hear from you. > > Best Regards, > Shuai Wu. + centos-gsoc ML. As it will help others who want to know about the topic. Hi Shuai, Thanks for your interest in the topic. Here are the below things/pointers for you to start. * Learn about GlusterFS o GlusterFS Admin Guide : https://github.com/gluster/glusterfs/tree/master/doc/admin-guide/en-US/mark… o http://wiki.centos.org/SpecialInterestGroup/Storage/gluster-Quickstart o http://www.gluster.org/ * Learn about qcow2 images, open stack images and AMI images o http://en.wikibooks.org/wiki/QEMU/Images o http://docs.openstack.org/image-guide/content/ o http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html Let me know if you need anything else. I am available in #gluster-dev, #gluster or #centos-devel (Freenode). Feel free to ping me about the topics or any information. Thanks, Lala

1 0

Re: [CentOS-gsoc] Regarding GSoC Idea: CentOS Cloud Storage components
by Lalatendu Mohanty 11 Mar '15

11 Mar '15

On 03/07/2015 12:16 PM, Ashish Agarwal wrote: > Respected Sir, > > I am writing to you to explore the possibility of a research project > under your esteemed guidance in GSoC. > > I have worked on automatic deployment of virtual labs over cloud > infrastructure. I captured the essence of cloud computing and learnt > how virtualization works. The research involved implementation of a > python API to provide a thin abstraction layer on top of the available > cloud interfaces and set up experiments on cloud infrastructures > (OpenStack and Eucalyptus). > > I have a strong belief in open source technology and free availability > of knowledge. I believe that the knowledge in meant to be shared and > spread. Hence I have experience working with Version Control Systems > (GIT). I am very much versed with languages like python, c++ and cloud > infrastructures like OpenStack, Eucalyptus. > > Thank you for your consideration. In anticipation of a positive response. > > Yours sincerely, > > Ashish Agarwal +centos-gsoc mailing list. Hi Ashish, Thanks for you interest in the topic. As per GSoC schedule [1] you have time till March 16 to understand the requirements. So please go ahead and let us know if you have questions to understand the requirement for the task. When the application period opens, we will use the Melange tool [1] for tracking the application and associated discussions. [1] https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2… Thanks, Lala

1 0

gsoc 2015 projects
by Marios-Evaggelos Kogias 11 Mar '15

11 Mar '15

Dear CentOS gsoc mentors, my name is Marios and I am a student from Greece. I participated in gsoc last year in LTTng project and I am interested in participating again this year in the CentOS project. I found the projects you proposed extremely interesting and I am really confused which one to choose to make my proposal for. So, I would like to ask some clarifying questions, which are groupped per project. I address my questions here but I am more than willing to talk with the equivalent mentors in IRC or mail. My questions are: * Centos cloud storage - Is glusterfs a prerequisite for the program? Is it possible to use another fs, eg CephFS, because I am familiar with the Ceph architecture, since I have partially worked with that. * Lightweight Cloud instance contextualisation tool Can I have some more details about this project especially as far as the lightwieght part is concerned. As I imagine this tool will not be CentOS specific, right? * Xen based Hypervisor in a Box According to the ideas page "the aim of this task is to build an installer instance". So, will the result be like a live CD where the user can try different CentOS versions, or a new instance that from the time is installed xen is running and several CentOS VMs can be initiated. * App to trim containers Is this project about docker containers or containers in general? Are there any specific ideas and tools in mind behind that project or it is more of a research project? * Cloud in a box Is there any prerequisite about the cloud software that should be used? There are a lot of choices eg OpenStack, oVirt, synnefo As you can understand from my questions I am more interested in cloud and virtualization projects and CentOS is among the few gsoc projects that offer me this chance. I am looking forward your reply. Best regards, Marios

2 1

Develop a kpatch delivery mechanism
by vladiksonic＠mail.ru 10 Mar '15

10 Mar '15

Hello again! I am not very familiar with technologies used in repos, thus I have some questions: 1) I would like to use nodejs to manage the repo. Is that possible? Or is there some other technology already used for doing that kind of job? (can't find it in google, I am ready to read about it, so please, attach a link to that if possible) 2) What exactly is hard about metafile? As far as I understood, there is already a format used by yum, and it already has quite some info we need to load the patches. Maybe we can employ it in its exact form + some alternations/additions? (https://en.opensuse.org/openSUSE:Standards_Rpm_Metadata) - this will totally kill a need to write an own parser - it can be done by nearly copying yum's code for that, and will se something familiar for developers, which increases manageability. 3) I think CLI can be created by writing a yum plugin. Or is there some problems I do not know about with that? 4) What is used on the backend for managing repos? (I probably will need that experience when writing one for patches). 5) I am right now working on a starup, and I will have my exams in June, so I probably will start coding earlier, Hope that is not a problem :) PS. I wrote a message asking about "Extendable system hardening tool" earlier I am deciding between the two, and I would prefer the first one, but if that won't be possible, well - I am eager to contribute to this one, so I ask beforhand. Also, maybe this will be useful for someone else, if I can eventually take the first project, or if I do not pass at all (hope that does not happen, though) I'm looking forward to hearing from you, Vladislav Babkin

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

gsoc March 2015