devel March 2007

devel@lists.centos.org

64 participants
39 discussions

Malloc behavior
by Leonardo Vilela Pinheiro 11 Mar '07

11 Mar '07

Hi, I'm sorry for a so big email, but this might be a bug. As some of you may know, there is a characteristic of malloc called overcommit, which lets a program "allocate" more virtual memory than is available on the system. I've set overcommit_memory=2 and turned off swap, but I believe the behaviour of malloc is not the expected. Let me introduce the overcommit, then show the test code, then the results. //-- Starting introduction of overcommit According to http://www.die.net/doc/linux/man/man3/malloc.3.html: "By default, Linux follows an optimistic memory allocation strategy. This means that when *malloc*() returns non-NULL there is no guarantee that the memory really is available. This is a really bad bug. In case it turns out that the system is out of memory, one or more processes will be killed by the infamous OOM killer. In case Linux is employed under circumstances where it would be less desirable to suddenly lose some randomly picked processes, and moreover the kernel version is sufficiently recent, one can switch off this overcommitting behavior using a command like # echo 2 > /proc/sys/vm/overcommit_memorySee also the kernel Documentation directory, files *vm/overcommit-accounting* and *sysctl/vm.txt*." Looking at vm/overcommit-accounting: "The Linux kernel supports the following overcommit handling modes 0 - Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage. root is allowed to allocate slightly more memory in this mode. This is the default. 1 - Always overcommit. Appropriate for some scientific applications. 2 - Don't overcommit. The total address space commit for the system is not permitted to exceed swap + a configurable percentage (default is 50) of physical RAM. Depending on the percentage you use, in most situations this means a process will not be killed while accessing pages but will receive errors on memory allocation as appropriate. The overcommit policy is set via the sysctl `vm.overcommit_memory'. The overcommit percentage is set via `vm.overcommit_ratio'." //-- Ending introduction of overcommit Then I started doing some tests. Below is my the test code: //---------------------------- Top of test code #include <stdio.h> #include <malloc.h> #include <limits.h> #include <string.h> #include <unistd.h> int main(void) { int i; char *p; 4 char *r = malloc(1048576); // Filling some useless space to copy later for (i = 0; i <= 1048576; i++) { *(char*)(r+i) = i; } for (i = 1; ; i++) { p = malloc(1048576); if (p == NULL) { printf("malloc %d MB - %p\n", i, p); printf("Returned null\n"); return 0; } else { // printf("malloc %d MB - %p\n", i, p); if (memcpy(p, r, 1048576) == 0) { printf("Error in memcpy\n"); } usleep(5); } } printf("Ok, no NULL\n"); return 0; } //---------------------------- End of test code Conditions: Kernel: 2.6.9-42.0.10.ELsmp RAM memory: 2GB Swap: OFF overcommit_memory=2 (changed by me) overcommit_ratio=50 (default) ' free ' yields: total used free shared buffers cached Mem: 2065252 602872 1462380 0 12372 283248 -/+ buffers/cache: 307252 1758000 Swap: 0 0 0 There is about 1.4GB free RAM memory. Then I've executed the test code got: malloc 178 MB - (nil) Returned null According to the kernel document above: "The total address space commit for the system is not permitted to exceed swap + a configurable percentage (default is 50) of physical RAM. Depending on the percentage you use, in most situations this means a process will not be killed while accessing pages but will receive errors on memory allocation as appropriate." Shouldn't I have got something about 1000 MB instead of 178 MB, because 1000 MB is 50% of physical memory? As long as I have 1.4GB free, I should be able to allocate those 1000 MB. Are there other (configurable or not) limits for memory allocation? (ulimit is not being used) Then I repeated the same test, but with swap on. Swap size is 1GB. Conditions: Kernel: 2.6.9-42.0.10.ELsmp RAM memory: 2GB Swap: 1GB overcommit_memory=2 (changed by me) overcommit_ratio=50 (default) ' free ' yields: total used free shared buffers cached Mem: 2065252 605480 1459772 0 12628 283512 -/+ buffers/cache: 309340 1755912 Swap: 1048568 0 1048568 Then I've executde the test code and got: malloc 1198 MB - (nil) Returned null It means this time I've allocated successfuly 1198 MB. Shouldn't I have got something about 2GB? Thanks Leonardo Pinheiro

2 1

Building perl (and perl threads memory leak)...
by James Olin Oden 11 Mar '07

11 Mar '07

Hi All, I'm trying to rebuild the version of perl that is in CentOS 4.3 to update the version of threads::shared(3pm) that has a serious memory leak in it. When I try to build it some of the tests its running fail (failure went off my screen but I'll have that in a moment) and it failed to build. BTW, I'm building on a CentOS 4.2 system. So my questions are two fold: - Does anyone know of any known issues with building the perl SRPM in general? - Once I have the latest threads::shared in buildable SRPM is there an interest in picking that up, or is there anyone that would like to put it in a repository? Cheers...james

5 14

C5: yum-metadata and non-yum yum-metadata
by Karanbir Singh 11 Mar '07

11 Mar '07

firstly, we slipped on shipping beta out yesterday, but we hit a couple of issues that needed to be resolved before we could get the iso's out. Most of the things are now fixed and the final bits are building as I write this email. there is one issue that was raised by Johnny that really does need fixing, perhaps not something for beta-public, but something that does need to get fixed for Final release. <background> The way things stand right now, anaconda uses yum in c5, so the media now contains a local repository metadata but its not the standard meta data used by yum. Anaconda and pirut need a specific string added in ( the mediaid fluffage, some of the fedora guys will be familiar with ) so that they can use the media at installtime and also be able to use the same media post install to add packages. The issue is, that this now creates a problem in that you can no longer loop mount the DVD iso under a http/ftp server and just use that as a remote repository to pull packages from using yum. </background> So now the thing is - for us to facilitate this there are three options. 1. We ship 2 repodata/'s on the media and use one for anaconda and use the other for yum ( ugly fix ) 2. We change anaconda to not need that media stuff in the repodata/, and get that from somewhere else ( not gonna happen : too much work, and too much potential to change from upstream in ways not even imaginable ) 3. We fix yum to use the media fluff'd repodata/ ( potentially a few lines fix and should just work ). This will allow yum + anaconda, pirut and friends to essentially just use the same repository metadata. What does everyone else think about this ? I think from this email, my own personal preference is easy to make out. Also, is this really an issue we even need to worry about ? For me, its an issue since I use loop mounted media right off the block. Does anyone else do the same, or is everyone just coping the tree's out and doing createrepo foo ? -- Karanbir Singh : http://www.karan.org/ : 2522219@icq

4 6

Re: [CentOS] RHWAS rebuild efforts
by Karanbir Singh 10 Mar '07

10 Mar '07

Leonardo Vilela Pinheiro wrote: > Maybe this thread should go to centos-devel list. sure, and moved! > Can you give us more details on what packages need JDK, and which JDK? > I'd like to understand what is going on, which packages are not ported > to Centosplus yet and why. There are bits of jboss and mod_jk stuff form rhwas4as that need the java stuff. > Do we have to build (compile) JDK? if you want to, but considering its unlikely we will get the sources for 1.5.x jdk, I'd say we might need the route of nosrc.rpm instead :) > Can't we use JPackage.org's and give users instructions on how to > install and enable their repository? Maybe thats possible ( I know at least the mysql-jdbc package has its origins in jpp ) , would you like to take this effort on ? Jim and HughesJr have been looking at the rhwas stuff, if you are interested in taking this up, maybe we can start a wiki page with details. - KB -- Karanbir Singh : http://www.karan.org/ : 2522219@icq

1 0

Octave for CentOS5
by Phil Schaffner 08 Mar '07

08 Mar '07

Apparently Octave has been orphaned by the upstream for EL5 , although it is still in Fedora Extras. It does successfully build from FC6/7 src.rpm packages under CentOS5 beta, albeit with a few additional requirements... SRPMS: fftw-3.1.2-3.fc6.src.rpm glpk-4.13-1.fc6.src.rpm hdf5-1.6.5-7.fc7.src.rpm octave-2.9.9-1.fc6.src.rpm ufsparse-2.1.1-1.fc6.src.rpm RPMS: fftw-3.1.2-3.el5.i386.rpm fftw-debuginfo-3.1.2-3.el5.i386.rpm fftw-devel-3.1.2-3.el5.i386.rpm glpk-debuginfo-4.13-1.el5.i386.rpm glpk-devel-4.13-1.el5.i386.rpm glpk-utils-4.13-1.el5.i386.rpm hdf5-1.6.5-7.el5.i386.rpm hdf5-debuginfo-1.6.5-7.el5.i386.rpm hdf5-devel-1.6.5-7.el5.i386.rpm octave-2.9.9-1.el5.i386.rpm octave-debuginfo-2.9.9-1.el5.i386.rpm octave-devel-2.9.9-1.el5.i386.rpm ufsparse-2.1.1-1.el5.i386.rpm ufsparse-debuginfo-2.1.1-1.el5.i386.rpm ufsparse-devel-2.1.1-1.el5.i386.rpm Is there interest in adding this to centosplus? Or perhaps Dag or Karanbir will pick it up? I will be building it locally anyway, if it is not available elsewhere, so could help with keeping updated. Phil

13 30

Centosplus release and updates cycle
by Leonardo Vilela Pinheiro 08 Mar '07

08 Mar '07

Hi, I´d like to understand better the release cycle of Centosplus packages. >From http://mdw1982.com/pipermail/centos_search/2006-March/006556.html : "Packages for centosplus are submitted to the developers, we check package quality, build, stability, etc. We discuss the packages, and they go to the development repository. If they get enough positive feedback, they move to centosplus. If not, they sit there because ostensibly there is no one using/pushing for them." For every package in Centosplus, is there a responsible person in charge, which should release security updates/patches for it? Do you know how is this process inside Red Hat, and how our process compare to theirs? Thanks

2 1

CentOS 5 Beta Initial Test Release?
by Andreas Boschke 05 Mar '07

05 Mar '07

On Wed, 2007-02-21 at 13:14 +0000, Lance Davis wrote: > We are about to release an initial internal test release of CentOS 5 beta > to our qa team. > > If folks here want to be a part of that test they need to join the > centos-qa list (http://lists.centos.org/mailman/listinfo/centos- qa) and > send a message to this list (centos-devel) saying that > they would like to be a part of the qa process and join the qa team, > with details of relevant experience if you are not known to the core > developers. > > Membership of the centos-qa list will then be approved. Hi: I'm interested in CentOS 5 QA, and would appreciate membership into centos-qa. I've wrestled both willing and unwilling computers into running CentOS. Like any good systems administrator, rarely can our hundreds of users find me, nor do our computing services fail. Thanks, -Andreas Boschke -Senior Systems Administrator -Computer Science, UCSB -andreas(a)cs.ucsb.edu

1 0

CentOS 5 Beta Initial Test Release
by Lance Davis 05 Mar '07

05 Mar '07

We are about to release an initial internal test release of CentOS 5 beta to our qa team. If folks here want to be a part of that test they need to join the centos-qa list (http://lists.centos.org/mailman/listinfo/centos-qa) and send a message to this list (centos-devel) saying that they would like to be a part of the qa process and join the qa team, with details of relevant experience if you are not known to the core developers. Membership of the centos-qa list will then be approved. Lance -- uklinux.net - The ISP of choice for the discerning Linux user.

34 38

hwinfo
by Jordi Espasa Clofent 05 Mar '07

05 Mar '07

Hi folks, ¿Is there some intention to include hwinfo[*] package in some repo (ie, contrib or centosplus)? At present moment we've a lswh, which is an excellent tool, but I consider howinfo more powerful. In fact, Debian project has adapted it since 2004. Thanks.

1 0

Interested in Beta testing CentOS 5
by Nad 03 Mar '07

03 Mar '07

Hi I would be very interested in beta testing CentOS 5 and report any issues. I currently run shared hosting servers using Fedora and want to switch to CentOS. I would be more than happy to verify advanced configurations of hosting environments applications using bind/postfix/dovecot/squirelmail/mysql/perl/httpd/php etc. Sign me up for the QA. Cheers Nad

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

devel March 2007