Coming late to the meeting.. sorry :)
Ideally, I would ilke to see subscription-manager and python-rhsm
carried in CentOS. There is no need to make it part of @base as it is
upstream, but it would be nice to have them there.
-- bk
The previous brand-hunter.py let you provide package patterns. It would then search SRPM repositories to find SRPMs matching those patterns (using the magic of yum), download and install them, and search content (expanding gzip/bz2 files as it goes) for possible branding issues.
This version is a bit more modest. It assumes you have already downloaded
content, e.g. using git clone/get_sources.sh.
Then you run brand_hunter.py, providing the path to the directory (or
directories), and it searches files (still expanding zipped files as before).
---
brand_hunter.py | 153 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 153 insertions(+)
create mode 100755 brand_hunter.py
diff --git a/brand_hunter.py b/brand_hunter.py
new file mode 100755
index 0000000..64bbe9f
--- /dev/null
+++ b/brand_hunter.py
@@ -0,0 +1,153 @@
+#!/usr/bin/python
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>
+#
+
+"""
+brand_hunter
+
+searches spec and source files for possible Red Hat branding issues
+"""
+
+try:
+ import magic
+except ImportError as e:
+ raise ImportError("%s: please install the 'python-magic' package" % e)
+
+import os
+import re
+import sys
+import tarfile
+
+from optparse import OptionParser
+
+DEVNULL=open('/dev/null', 'w')
+
+MIME = magic.open(magic.MAGIC_MIME)
+MIME.load()
+
+RE = re.compile(r'[Rr][Ee][Dd]\s?[Hh][Aa][Tt]', flags=re.M)
+RE_EMAIL = re.compile(r'<[^@]+@redhat.com>', flags=re.M)
+
+TAR_OPEN_MODE = {
+ 'application/x-bzip2' : 'r:bz2',
+ 'application/x-gzip' : 'r:gz',
+ }
+
+
+def main(opts, args):
+ # validate directory
+ for topdir in args:
+ if not ('SPECS' in os.listdir(topdir) and 'SOURCES' in os.listdir(topdir)):
+ print ("Error: The specified directory does not contain SPECS and "
+ "SOURCES folders")
+ sys.exit(1)
+
+ # setup
+ print "\nprocessing %s" % topdir
+ issues_file = os.path.join(topdir, 'issues.txt')
+
+ # search files
+ issues = []
+ for dir in ['SPECS', 'SOURCES']:
+ searchdir = os.path.join(topdir, dir)
+ for path,_,files in os.walk(searchdir, topdown=False):
+ for file in files:
+ fp = os.path.join(path, file)
+ if opts.verbose: print fp
+ find_issues(issues, topdir, fp)
+
+ # output results
+ if issues:
+ with open(issues_file, 'w') as f:
+ f.write("\n".join(issues) + '\n')
+ print "- issues found: see %s" % issues_file
+ else:
+ print "- no issues found"
+
+ print "\n" # blank line at end for readability
+
+
+def find_issues(issues, topdir, file):
+ relpath = file.replace(topdir + '/', '')
+
+ if not os.path.exists(file):
+ return
+
+ mimetype=MIME.file(file)
+
+ if 'application/x-empty' in mimetype:
+ return
+
+ elif not 'charset=binary' in mimetype:
+ with open(file, 'r') as fo:
+ s = fo.read()
+ for m in RE.finditer(s):
+ lineno = s.count('\n',0,m.start())
+ line = s.split('\n')[lineno]
+
+ # filter out lines with email only matches
+ if opts.ignore_email:
+ email = RE_EMAIL.findall(line)
+ if email and len(email) == len(RE.findall(line)):
+ continue
+
+ issues.append('%s:%s:%s' % (relpath, lineno+1, line))
+
+ elif ('application/x-bzip2' in mimetype or
+ 'application/x-gzip' in mimetype):
+ try:
+ tf = tarfile.open(file, TAR_OPEN_MODE[mimetype.split(';')[0]])
+ except tarfile.ReadError:
+ # we can't decompress the file, so treat it as an unknown binary file
+ issues.append('%s:binary file' % relpath)
+ else:
+ tf.extractall(os.path.dirname(file))
+ os.remove(file)
+ for n in tf.getnames():
+ mp = os.path.join(os.path.dirname(file), n)
+ if not os.path.isdir(mp):
+ find_issues(issues, topdir, mp)
+
+ else:
+ issues.append('%s:binary file' % relpath)
+
+if __name__ == '__main__':
+
+ parser = OptionParser("usage: %prog [options] directory...",
+ description=(
+ "Searches SPEC and SOURCE folders in one or more directories for possible "
+ "Red Hat branding issues."
+ ))
+
+ parser.add_option('--ignore-email',
+ dest='ignore_email',
+ action='store_true',
+ default=False,
+ help="ignore text that matches '<email(a)redhat.com>'")
+
+ parser.add_option('--verbose', '-v',
+ dest='verbose',
+ action='store_true',
+ default=False,
+ help="print the names of files as they are processed")
+
+ opts,args = parser.parse_args(args=sys.argv[1:])
+
+ if not args:
+ print "Error: no directory specified"
+ sys.exit(1)
+
+ main(opts, args)
+ sys.exit()
--
1.8.3.1
In the expected place for srpms for RHEL7 there is a README
ftp.redhat.org:/redhat/linux/enterprise/7Server/en/os/README
It's contents are
----------------------------------------------------------------------------
Current sources for Red Hat Enterprise Linux 7 have been moved to the
following
location:
https://git.centos.org/project/rpms
----------------------------------------------------------------------------
Who/what is populating this area?
How are updates to packages handled? Do they go straight to
https://git.centos.org/project/rpms/ as the updates are published by
RedHat?
Does CentOS modify any of these packages?
Since it is implied that this "represents" the "srpm" for a given RHEL
package (given the above README from ftp.redhat.com) how do I know it has
not been tampered with?
-Connie Sieh
Hi
Yup, its that question again : do we want to leave, but
patch-to-non-functional the rhn and subscription-manager tools in the
distro. I know its useful to some projects to leave in, but do they care ?
- KB
--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
GnuPG Key : http://www.karan.org/publickey.asc
I had a look at http://buildlogs.centos.org/centos/7/os/x86_64-20140614/ but I could not find any qcow images for a VM guest.
Is it planned to produce one ?
Creating a cloud instance is our lowest effort way of testing new releases.
Thanks
Tim
As Karanbir announced it today, there will be "nightly builds" happening
every day, starting from today (the first one being scheduled to start
at 8PM UTC).
The whole process will be automated and would also start to reflect
those new trees.
That means that the url to enter for new network install, and yum
repositories will need to be using the 'latest' symlink when that one
will appear. (Normally full URL would be
http://buildlogs.centos.org/centos/7/os/x86_64-latest)
As it will be the first time we'll test the end-to-end automation
script, we have no real ETA, but we estimate the "time-to-process"
between 3 and 4 hours (so content would be available on
buildlogs.centos.org around 11 pm UTC, or slightly later)
We've also worked today on the Live Media iso images, so those ones will
also be pushed on buildlogs.centos.org (later today, during the "nightly
build" process) (see also
https://git.centos.org/summary/sig-core!livemedia.git)
As a reminder :
* RPM packages are still unsigned
* yum config files are still missing (but you can point yum to the
mentioned repository)
* that tree is still "not production" ready and still needs to be
investigated (branding hunt, anyone ?)
* we count on you to provide feedback (positive or negative, but
feedback requested please)
* use #centos-devel (irc.freenode.net) @centos-devel list (here), but
also file bugs reports on our bugs tracking system
(http://bugs.centos.org) and report those under the "CentOS-7" project
So , (ab)use the tree, break it, fix it (by filing bug reports - patches
*welcome* :-) )
--
Fabian Arrotin
gpg key: 56BEC54E | twitter: @arrfab
FYI: I've just filed bug # 7206 regarding a branding issue:
http://bugs.centos.org/view.php?id=7206
--
---- Fred Smith -- fredex(a)fcshome.stoneham.ma.us -----------------------------
The eyes of the Lord are everywhere,
keeping watch on the wicked and the good.
----------------------------- Proverbs 15:3 (niv) -----------------------------
Guillaume,
Don't see why not, sounds like the right place for it? I rebuilt a 6
kernel some while back for work that had MP-TCP and F-RTO in it along with
some twiddling of initRTO as well, those other two are in mainstream now
and the MP-TCP patch must have been fairly simple if I managed to work it
in. Shout if you need any help, I've got some doco somewhere.
> Message: 19
> Date: Sun, 15 Jun 2014 13:54:29 +0200
> From: Guillaume Derval <guillaume.derval(a)student.uclouvain.be>
> Subject: [CentOS-devel] Multipath-TCP in the centosplus kernel?
> To: centos-devel(a)centos.org
> Message-ID:
> <A8BF2232-7620-433F-82D0-C74FBEE5597B(a)student.uclouvain.be>
> Content-Type: text/plain; charset="us-ascii"
>
> Hello everyone,
>
> I am thinking about building custom kernels for CentOS-7 with a support
> for Multipath-TCP (http://multipath-tcp.org/) and I am wondering if a
> such heavy modification should have its place in the centosplus kernel?
>
> Modified kernel sources are available at
> https://github.com/multipath-tcp/mptcp and equivalent patches are there:
> http://multipath-tcp.org/patches/.
>
> Best regards,
> Guillaume Derval.
>
--
Regards,
Phil