Discuss

discuss@lists.centos.org

5 participants
34752 discussions

RE: Keeping data on 2 servers in sync !
by Jonathan Darton 10 Dec '05

10 Dec '05

On 12/7/05, Denis Croombs <denis(a)croombs.org> wrote: > On 12/6/05, Jonathan Darton <jdarton(a)ibigroup.com> wrote: > >> >I want to build 2 servers (both running samba) to provide file > >> >storage to 2 offices (approx 100 miles apart, linked via DSL) but all > >> >data writen to 1 server must also be saved to the other server. > >> >Both servers would also allow users to access the data via a VPN thus > >> >allowing 1 office with a failed server to access the other server via > >> >the vpn and still see the data from both offices. > >> >I currently have 1 server working but we want to add the second > >> >office to the system. (Currently 1 office has 10 users and the second > >> >office has 1 user connected via VPN ) but the second office will have > >> >20 within 12 > >> months > >> >and the first will have 35 soon )) > >> > >> >Has anyone done anything like this ? > >> > >> I am currently synchronizing multiple office locations using a program > >> called unison. Unison (http://www.cis.upenn.edu/~bcpierce/unison/) is > >> a very well written program that can perform 2 way file > >> synchronization. There are many configurable options with unison and I > recommend that you check it out. > >> In each office I have a PII350 128RAM Fedora or CentOS server running > >> unison and the files are accessed via samba. I also configure samba to > >> hide (veto) all of the temporary files used during synchronization. > >> For redundancy I place a slave server with each master server that > >> backs up all the user data / file system using rsync. This way if one > >> of my $5 PII servers catches fire I can automatically switch over with no > downtime for the users. > >> > >> The only downfall I have encountered is with Autocad files not > >> properly reading the synchronized .dwl lock file and more than one > >> user working on the same file. As a work around for this I have > >> configured Unison to keep a backup of the last 20 versions of a file. > >> This way I can always hit my backups to retreive lost data. As a side > >> note, if anyone knows a work around for the stubborn autocad dwl lock > file let me know :))! > >> > >> In any case my implementation has allowed me to synchronize file > >> systems between 4 offices (3 in Canada, 1 in USA), using recycled > >> hardware that was otherwise going to be donated/trashed. > >> > >> Let me know if you have any further questions. > > > >I'm about to do a Unison setup on two CentOS servers, so I'm thrilled to > see this response. I also work with Architects >>sometimes, so I'm > interested to hear about the dwl lock file issue. > > > >My one compound question: how are you invoking Unison? In batch mode, with > cron? How often? Wat other options did you > >consider before settling on the scheme you use? > > I see the project is no longer supported, do you have rpms for it ? > > Thanks >It's not that it's no longer supported, really, just that the project >that initiated it is not an officially-funded academic project. It has >actually been updated since the termination of the project, it looks >like. >I haven't installed it on CentOS yet but I got FC4 rpms from fedora >extras (I just typed 'yum install unison' and it worked). I think it's >in pretty common usage in RHEL too, so it can't be too hard to find >for CentOS, right? (But let us know what you turn up :-)). I created a script that is run by cron every minute. The script creates a lock file for the session, rsyncs the changes to the backup server, synchronizes offices using unison, deletes the lock file. Unison is executed in "batch" and "silent" mode so that no output is generated and mailed by cron. One other thing you need to consider is the size of the file system you are trying to synchronize. I have a very large directory structure in two offices which required running unison a second time splitting up the synchronization into two sessions. This can easily be done by ignoring a certain path/paths and then synchronizing them in the next session. If you are thinking of synchronizing between more than 2 locations, you need to plan the fastest route (ie. A<->B<->C or B<->A<->C). I have had no problems synchronizing between more than two locations. For all of my implementations using Fedora or CentOS operating systems I have downloaded the unison source, installed the ocaml compiler (easy to find for centos/fedora use google) and compiled from source. I have never had a problem. Another benefit to the implementation that I have been using is the fact that employees in each office can work on files on the server as long as the internal network is functioning, regardless if the synchronization link / externel connection is working. The synchronization is basically transparent to the user, and if it happens to go down it will pick up where it left off as soon as it can regain a connection. Let me know if you require any further information. A. Jonathan Darton, B.COMM CIS System Consultant

8 13

perl module
by hiep＠ee.ucr.edu 09 Dec '05

09 Dec '05

Hi list, I have trouble install the following perl modules to my Centos 4.2 server and I need help Digest::SHA1 Digest::HMAC Net::DNS Time::HiRes HTML::Tagset HTML::Parser Pod::Usage Parse::Syslog Statistics::Distributions I tried "perl -MCPAN -e shell" and then "install Pod::Usage" and I got the following errors: CPAN: Storable loaded ok Going to read /root/.cpan/Metadata Warning: Found only 0 objects in /root/.cpan/Metadata CPAN: LWP::UserAgent loaded ok Fetching with LWP: ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz LWP failed with code[500] message[LWP::Protocol::MyFTP: Bad hostname 'ftp.perl.org'] Fetching with Net::FTP: ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz Trying with "/usr/bin/links -source" to get ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz ERROR at dump.c:188: Host not found System call "/usr/bin/links -source "ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz" > /root/.cpan/sources/authors/01mailrc.txt" returned status 1 (wstat 256) Warning: expected file [/root/.cpan/sources/authors/01mailrc.txt.gz] doesn't exist Trying with "/usr/bin/wget -O -" to get ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz --09:54:41-- ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz => `-' Resolving ftp.perl.org... failed: Name or service not known. System call "/usr/bin/wget -O - "ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz" > /root/.cpan/sources/authors/01mailrc.txt" returned status 1 (wstat 256) Warning: expected file [/root/.cpan/sources/authors/01mailrc.txt.gz] doesn't exist Issuing "/usr/kerberos/bin/ftp -n" ftp: ftp.perl.org: unknown host Not connected. Local directory now /root/.cpan/sources/authors Not connected. Not connected. Not connected. Not connected. Not connected. Not connected. Bad luck... Still failed! Can't access URL ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz. Please check, if the URLs I found in your configuration file () are valid. The urllist can be edited. E.g. with 'o conf urllist push ftp://myurl/' Could not fetch authors/01mailrc.txt.gz Fetching with LWP: ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz LWP failed with code[500] message[LWP::Protocol::MyFTP: Bad hostname 'ftp.perl.org'] Fetching with Net::FTP: ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz Trying with "/usr/bin/links -source" to get ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz ERROR at dump.c:188: Host not found System call "/usr/bin/links -source "ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz" > /root/.cpan/sources/modules/02packages.details.txt" returned status 1 (wstat 256) Warning: expected file [/root/.cpan/sources/modules/02packages.details.txt.gz] doesn't exist Trying with "/usr/bin/wget -O -" to get ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz --09:54:47-- ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz => `-' Resolving ftp.perl.org... failed: Name or service not known. System call "/usr/bin/wget -O - "ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz" > /root/.cpan/sources/modules/02packages.details.txt" returned status 1 (wstat 256) Warning: expected file [/root/.cpan/sources/modules/02packages.details.txt.gz] doesn't exist Issuing "/usr/kerberos/bin/ftp -n" ftp: ftp.perl.org: unknown host Not connected. Local directory now /root/.cpan/sources/modules Not connected. Not connected. Not connected. Not connected. Not connected. Not connected. Bad luck... Still failed! Can't access URL ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz. Please check, if the URLs I found in your configuration file () are valid. The urllist can be edited. E.g. with 'o conf urllist push ftp://myurl/' Could not fetch modules/02packages.details.txt.gz Fetching with LWP: ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz LWP failed with code[500] message[LWP::Protocol::MyFTP: Bad hostname 'ftp.perl.org'] Fetching with Net::FTP: ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz Trying with "/usr/bin/links -source" to get ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz ERROR at dump.c:188: Host not found System call "/usr/bin/links -source "ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz" > /root/.cpan/sources/modules/03modlist.data" returned status 1 (wstat 256) Warning: expected file [/root/.cpan/sources/modules/03modlist.data.gz] doesn't exist Trying with "/usr/bin/wget -O -" to get ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz --09:54:53-- ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz => `-' Resolving ftp.perl.org... failed: Name or service not known. System call "/usr/bin/wget -O - "ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz" > /root/.cpan/sources/modules/03modlist.data" returned status 1 (wstat 256) Warning: expected file [/root/.cpan/sources/modules/03modlist.data.gz] doesn't exist Issuing "/usr/kerberos/bin/ftp -n" ftp: ftp.perl.org: unknown host Not connected. Local directory now /root/.cpan/sources/modules Not connected. Not connected. Not connected. Not connected. Not connected. Not connected. Bad luck... Still failed! Can't access URL ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz. Please check, if the URLs I found in your configuration file () are valid. The urllist can be edited. E.g. with 'o conf urllist push ftp://myurl/' Could not fetch modules/03modlist.data.gz Going to write /root/.cpan/Metadata Warning: Cannot install Pod::Usage, don't know what it is. Try the command i /Pod::Usage/ to find objects with matching identifiers. Then I tried to download individual module and install it manually, but I got dependency warnings. Please help!!! Thanks. JC

10 13

Manual update
by Jean Lee 09 Dec '05

09 Dec '05

Hi all, I have a problem :). On my CentOS workstation, I had glib2-2.4.7-1 installed. This is the last version which is provided by the CentOS distribution However, I need to install a software which requires an higher glib2 version so : I took the source of glib2-2.8.4-1 on the gtk+ web site and creates the rpm installation file with "chkinstall" I obtained the glib2-2.8.4-1.i386.rpm file Then I execute yum to update the glib2 package : yum localupdate glib2-2.8.4-1.i386.rpm And know, I have a lot of problems. Surely because some library files where overwritten with the newer version files (Right or wrong ?).... Is there a way to recover the problem ? (Now, I can't launch a terminal under gnome) Well, I have broken the distribution and I feel that I will have to reinstall it. For the next time that I will try to update a package manually : Is there a safe way to update manually the glib2 library without crashing all the distribution ? Or is there a way to install the latest version of glib2 in order to have the two libraries versions installed (glib2-2.4.7-1 and glib2-2.8.4-1)? or something else ????? Thank you for your answers, Jean LEE

3 2

Hald
by Sam Drinkard 09 Dec '05

09 Dec '05

No manpage or other info in the system about the hald or hal. What little I did find on a RH site did not really enlighten me that much. Is there some obscure documentation somewhere about the hald and its assets and attributes? -- Snowman

2 1

CPU usage from top
by Bowie Bailey 09 Dec '05

09 Dec '05

I'm seeing some strange results from top on CentOS 4.0. When it first starts, it reports low CPU usage. The next refresh (and subsequent) refreshes, it shows high CPU usage. In both cases, there is nothing shown in the process list that is using much CPU. What is causing this? Here is some truncated output from a single run of "top -b -n 2": ---------------------------------------------------------------------------- -- top - 10:22:02 up 34 days, 14:36, 2 users, load average: 0.98, 0.56, 0.51 Tasks: 116 total, 1 running, 115 sleeping, 0 stopped, 0 zombie Cpu(s): 3.7% us, 0.7% sy, 0.0% ni, 94.2% id, 1.3% wa, 0.0% hi, 0.0% si Mem: 1034676k total, 610804k used, 423872k free, 35796k buffers Swap: 1052248k total, 14220k used, 1038028k free, 314524k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16412 root 16 0 2280 864 664 R 5.6 0.1 0:00.06 top 1 root 16 0 3148 516 464 S 0.0 0.0 0:01.91 init 2 root RT 0 0 0 0 S 0.0 0.0 0:04.32 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.34 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:02.32 migration/1 5 root 34 19 0 0 0 S 0.0 0.0 0:00.39 ksoftirqd/1 6 root 5 -10 0 0 0 S 0.0 0.0 0:00.13 events/0 7 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/1 8 root 5 -10 0 0 0 S 0.0 0.0 0:00.18 khelper 9 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/0 top - 10:22:05 up 34 days, 14:36, 2 users, load average: 0.98, 0.56, 0.51 Tasks: 116 total, 1 running, 115 sleeping, 0 stopped, 0 zombie Cpu(s): 49.1% us, 1.6% sy, 0.0% ni, 48.8% id, 0.2% wa, 0.2% hi, 0.2% si Mem: 1034676k total, 610804k used, 423872k free, 35800k buffers Swap: 1052248k total, 14220k used, 1038028k free, 314520k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16412 root 17 0 2288 960 748 R 1.6 0.1 0:00.11 top 16052 bowieb 16 0 8288 2232 1812 S 0.3 0.2 0:03.04 sshd 1 root 16 0 3148 516 464 S 0.0 0.0 0:01.91 init 2 root RT 0 0 0 0 S 0.0 0.0 0:04.32 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.34 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:02.32 migration/1 5 root 34 19 0 0 0 S 0.0 0.0 0:00.39 ksoftirqd/1 6 root 5 -10 0 0 0 S 0.0 0.0 0:00.13 events/0 7 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/1 8 root 5 -10 0 0 0 S 0.0 0.0 0:00.18 khelper 9 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/0 ---------------------------------------------------------------------------- -- Notice 3.7% usage on the first one and 49.1% usage on the second. I suspect that top isn't showing me everything. I know that some versions of top had problems with showing proper CPU usage for threaded applications. Is that still a problem with the current version of top? Bowie

2 2

RE: [CentOS] cs & gfs
by Bowie Bailey 09 Dec '05

09 Dec '05

From: Karanbir Singh [mailto:mail-lists@karan.org] > > Bowie Bailey wrote: > > From: Karanbir Singh [mailto:mail-lists@karan.org] > > > >>Jure Pecar wrote: > >> > >>>On Mon, 21 Nov 2005 08:41:19 -0600 > >>>Johnny Hughes <mailing-lists(a)hughesjr.com> wrote: > >>> > >>> > >>>>We just built some more i686 packages that Karanbir is going to test in > >>>>the next day or so. > >>>> > >>>>If they work even fairly well, we will make them available for testing. > >>> > >>>Any progress on this? > >> > >> http://lists.centos.org/pipermail/centos-announce/2005-December/000946.html > > > > I don't remember seeing this. Is the announce list mirrored to this > > one, or do I need to subscribe to yet another mailing list to get > > these announcements? > > the annouce list is posted in digest form to this list, one post per > day , on days when there is a post in the announce list. Ok, I guess I just missed this one. On a question more related to the original subject... I have a server using cs and gfs that is built from the source rpms on http://rpm.karan.org/el4/csgfs/SRPMS/ (modified only enough to compile on the latest kernel without hugemem). Is it safe (or even possible) to update these from the official distribution, or do I need to remove the current stuff and rebuild the cluster using the official rpms? I would like to get to a point where this can be kept up to date with a 'yum update' command now that it's in the CentOS distribution. Bowie

2 1

To add or to reinstall - 4.2
by Robert Moskowitz 09 Dec '05

09 Dec '05

That is the question. I don't know what I did NOT accept, but Firefox does not seem to be installed on a recently set up server. So is there some easy way to go through the install stuff from CDs to add this or do I punt? Then I have to reread the stuff to set up YUM again.

2 1

memory problems
by Tom De Blende 09 Dec '05

09 Dec '05

Hi, Since we upgraded our server to 3.6, it runs out of memory. This is what I can find in syslog (multiple similar entries): Dec 7 19:52:42 gcclo77 kernel: Out of Memory: Killed process 31375 (nagios). Dec 7 19:52:43 gcclo77 kernel: Mem-info: Dec 7 19:52:43 gcclo77 kernel: Zone:DMA freepages: 2927 min: 0 low: 0 high: 0 Dec 7 19:52:43 gcclo77 kernel: Zone:Normal freepages: 737 min: 766 low: 4031 high: 5791 Dec 7 19:52:43 gcclo77 kernel: Zone:HighMem freepages: 126 min: 255 low: 510 high: 765 Dec 7 19:52:43 gcclo77 kernel: Free pages: 3790 ( 126 HighMem) Dec 7 19:52:43 gcclo77 kernel: ( Active: 162420/241, inactive_laundry: 0, inactive_clean: 0, free: 3790 ) Dec 7 19:52:43 gcclo77 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2927 Dec 7 19:52:43 gcclo77 kernel: aa:157445 ac:1554 id:301 il:0 ic:0 fr:737 Dec 7 19:52:43 gcclo77 kernel: aa:2791 ac:569 id:1 il:0 ic:0 fr:126 Dec 7 19:52:43 gcclo77 kernel: 3*4kB 2*8kB 4*16kB 3*32kB 4*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 11708kB) Dec 7 19:52:43 gcclo77 kernel: 49*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2948kB) Dec 7 19:52:43 gcclo77 kernel: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 504kB) Dec 7 19:52:44 gcclo77 kernel: Swap cache: add 861742, delete 858950, find 7617227/7652707, race 0+36 Dec 7 19:52:44 gcclo77 kernel: 13082 pages of slabcache Dec 7 19:52:44 gcclo77 kernel: 9852 pages of kernel stacks Dec 7 19:52:44 gcclo77 kernel: 17191 lowmem pagetables, 28807 highmem pagetables Dec 7 19:52:44 gcclo77 kernel: 32 bounce buffer pages, 32 are on the emergency list Dec 7 19:52:44 gcclo77 kernel: Free swap: 0kB Dec 7 19:52:44 gcclo77 kernel: 262138 pages of RAM Dec 7 19:52:44 gcclo77 kernel: 32762 pages of HIGHMEM Dec 7 19:52:44 gcclo77 kernel: 5784 reserved pages Dec 7 19:52:44 gcclo77 kernel: 7450 pages shared Dec 7 19:52:45 gcclo77 kernel: 2792 pages swap cached Dec 7 19:52:45 gcclo77 kernel: Out of Memory: Killed process 31401 (nagios). Dec 7 19:52:45 gcclo77 kernel: Mem-info: Dec 7 19:52:45 gcclo77 kernel: Zone:DMA freepages: 2927 min: 0 low: 0 high: 0 Dec 7 19:52:45 gcclo77 kernel: Zone:Normal freepages: 753 min: 766 low: 4031 high: 5791 Dec 7 19:52:45 gcclo77 kernel: Zone:HighMem freepages: 126 min: 255 low: 510 high: 765 Dec 7 19:52:45 gcclo77 kernel: Free pages: 3806 ( 126 HighMem) Dec 7 19:52:45 gcclo77 kernel: ( Active: 162136/453, inactive_laundry: 72, inactive_clean: 0, free: 3806 ) Dec 7 19:52:45 gcclo77 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2927 Dec 7 19:52:45 gcclo77 kernel: aa:157502 ac:1554 id:172 il:72 ic:0 fr:753 Dec 7 19:52:46 gcclo77 kernel: aa:2666 ac:439 id:256 il:0 ic:0 fr:126 Dec 7 19:52:46 gcclo77 kernel: 3*4kB 2*8kB 4*16kB 3*32kB 4*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 11708kB) Dec 7 19:52:46 gcclo77 kernel: 65*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 3012kB) Dec 7 19:52:46 gcclo77 kernel: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 504kB) Dec 7 19:52:46 gcclo77 kernel: Swap cache: add 861742, delete 858950, find 7617227/7652707, race 0+36 Dec 7 19:52:46 gcclo77 kernel: 13082 pages of slabcache Dec 7 19:52:46 gcclo77 kernel: 9852 pages of kernel stacks Dec 7 19:52:46 gcclo77 kernel: 17178 lowmem pagetables, 28807 highmem pagetables Dec 7 19:52:46 gcclo77 kernel: 32 bounce buffer pages, 32 are on the emergency list Dec 7 19:52:47 gcclo77 kernel: Free swap: 0kB Dec 7 19:52:47 gcclo77 kernel: 262138 pages of RAM Dec 7 19:52:47 gcclo77 kernel: 32762 pages of HIGHMEM Dec 7 19:52:47 gcclo77 kernel: 5784 reserved pages Dec 7 19:52:47 gcclo77 kernel: 7020 pages shared Dec 7 19:52:47 gcclo77 kernel: 2792 pages swap cached Anyone any ideas as to what causes this? Thanks! Tom

1 0

RE: [CentOS] cs & gfs
by Bowie Bailey 09 Dec '05

09 Dec '05

From: Karanbir Singh [mailto:mail-lists@karan.org] > > Jure Pecar wrote: > > On Mon, 21 Nov 2005 08:41:19 -0600 > > Johnny Hughes <mailing-lists(a)hughesjr.com> wrote: > > > >>We just built some more i686 packages that Karanbir is going to test in > >>the next day or so. > >> > >>If they work even fairly well, we will make them available for testing. > > > > Any progress on this? > > http://lists.centos.org/pipermail/centos-announce/2005-December/000946.html I don't remember seeing this. Is the announce list mirrored to this one, or do I need to subscribe to yet another mailing list to get these announcements? Bowie

2 1

Server displaying ?
by scott 09 Dec '05

09 Dec '05

I figure this is an apache issue, but I dont' know where to look. Since moving my website from an older Redhat installation to my new Centos 4 server (all updates installed), all my web pages display a "?" anywhere I have a non-breaking space. If I do a view source in my browser I see the " " character, but when looking at it on my server via ssh, I see a vertical bar, similar, but not a pipe. And when looking at the source on a different browser I see a ASCII 160 character. But on all browsers, I'm seeing a ? on the display, instead of the non-breaking space. Anyone know what I need to configure/correct/change in Apache to correct this? Thanks, Scott

3 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss