Well, yesterday my 5.1 and 4.6 systems could connect to
torrent.centos.org
OK. Today neither system can connect. I can ping, but that's all. No changes were made to the firewall during this time.
Any tips on debugging this?
I successfully http downloaded all the ISO images from various mirrors this morning, so I think my net and firewall are OK.
TIA
William L. Maltby wrote:
Well, yesterday my 5.1 and 4.6 systems could connect to
torrent.centos.org
OK. Today neither system can connect. I can ping, but that's all. No changes were made to the firewall during this time.
Any tips on debugging this?
Is Comcast your Internet provider?
On Tue, 2007-12-18 at 15:06 -0800, Florin Andrei wrote:
William L. Maltby wrote:
Well, yesterday my 5.1 and 4.6 systems could connect to
torrent.centos.org
OK. Today neither system can connect. I can ping, but that's all. No changes were made to the firewall during this time.
Any tips on debugging this?
Is Comcast your Internet provider?
No. Almost as bad? Time-Warner cable. But I suspect it's not them. Yesterday was OK, today no. Also, for the 5.0 amd 4.5 releases, I was able to download and share the stuff via torrent.
I guess they could have changed there blockings since then though.
I'm running a port scan right now. So far, http and rsync are the only porst found open. The torrent file show port 6969 - but I don't know for sure if that represents the source port or a port I'm supposed to use because I don't have the layout of the torrent file. But I do see this in the torrent file:
d8:announce39:http://torrent.centos.org:6969/announce13: ...
So I'm betting that's the source port.
Thanks for replying,
William L. Maltby wrote:
I'm running a port scan right now. So far, http and rsync are the only porst found open. The torrent file show port 6969 - but I don't know for sure if that represents the source port or a port I'm supposed to use because I don't have the layout of the torrent file. But I do see this in the torrent file:
d8:announce39:http://torrent.centos.org:6969/announce13: ...
your torrent client can use any port it wants to, it has to be able to accept connections on that port, so whatever port you configure it for, you enable in your firewall, (or port forward if you have a consumer internet sharing appliance router)
the port in the torrent file is the tracker port, this is seperate.
On Tue, 2007-12-18 at 15:46 -0800, John R Pierce wrote:
William L. Maltby wrote:
I'm running a port scan right now. So far, http and rsync are the only porst found open. The torrent file show port 6969 - but I don't know for sure if that represents the source port or a port I'm supposed to use because I don't have the layout of the torrent file. But I do see this in the torrent file:
d8:announce39:http://torrent.centos.org:6969/announce13: ...your torrent client can use any port it wants to, it has to be able to accept connections on that port, so whatever port you configure it for, you enable in your firewall, (or port forward if you have a consumer internet sharing appliance router)
I've got IPCop as my firewall/router, all default configs, no DMZ, just internet side and private side.
I've not found a need to modify its settings in the past.
I use rtorrent and see it using ports 6490 (IIRC) and I get messages saying I can't connect to the tracker. I don't even know if this is important yet, but I'm assuming it provides some benefit to myself and/or others since it seems to try to connect to it by default.
the port in the torrent file is the tracker port, this is seperate.
Thanks for the start of education. If perused some of the docs at the torrent site, but know very little as yet.
<snip sig stuff>
Thanks again,
William L. Maltby wrote:
On Tue, 2007-12-18 at 15:46 -0800, John R Pierce wrote:
William L. Maltby wrote:
I'm running a port scan right now. So far, http and rsync are the only porst found open. The torrent file show port 6969 - but I don't know for sure if that represents the source port or a port I'm supposed to use because I don't have the layout of the torrent file. But I do see this in the torrent file:
d8:announce39:http://torrent.centos.org:6969/announce13: ...
ah. lets backup.
# nmap -sT -p6969 -vv torrent.centos.org ... Interesting ports on 66.147.238.146: PORT STATE SERVICE 6969/tcp closed acmsoda
...
appears the tracker is down.
On Tue, 2007-12-18 at 16:10 -0800, John R Pierce wrote:
William L. Maltby wrote:
On Tue, 2007-12-18 at 15:46 -0800, John R Pierce wrote:
William L. Maltby wrote:
I'm running a port scan right now. So far, http and rsync are the only porst found open. The torrent file show port 6969 - but I don't know for sure if that represents the source port or a port I'm supposed to use because I don't have the layout of the torrent file. But I do see this in the torrent file:
d8:announce39:http://torrent.centos.org:6969/announce13: ...ah. lets backup.
# nmap -sT -p6969 -vv torrent.centos.org ... Interesting ports on 66.147.238.146: PORT STATE SERVICE 6969/tcp closed acmsoda
...
appears the tracker is down.
OK. I can stop sweating that I screwed something up and will have to dig much deeper into things with which I have only passing familiarity.
I ran your command and got the same results.
nmap -sT -p6969 -vv torrent.centos.org
Starting nmap 3.70 ( http://www.insecure.org/nmap/ ) at 2007-12-18 19:19 EST Machine 66.147.238.146 MIGHT actually be listening on probe port 80 Initiating Connect() Scan against 66.147.238.146 [1 port] at 19:19 The Connect() Scan took 0.04s to scan 1 total ports. Host 66.147.238.146 appears to be up ... good. Interesting ports on 66.147.238.146: PORT STATE SERVICE 6969/tcp closed acmsoda
Nmap run completed -- 1 IP address (1 host up) scanned in 0.453 seconds
<snip sig stuff>
Thanks again for the help. I'm gonna stash this command in my local bin and RTFM on that. BTW, the port scan also did not show the port open. But it sure took awhile, checking all those other ports too.
Thanks again,
John R Pierce wrote:
William L. Maltby wrote:
On Tue, 2007-12-18 at 15:46 -0800, John R Pierce wrote:
William L. Maltby wrote:
I'm running a port scan right now. So far, http and rsync are the only porst found open. The torrent file show port 6969 - but I don't know for sure if that represents the source port or a port I'm supposed to use because I don't have the layout of the torrent file. But I do see this in the torrent file:
d8:announce39:http://torrent.centos.org:6969/announce13: ...ah. lets backup.
# nmap -sT -p6969 -vv torrent.centos.org ... Interesting ports on 66.147.238.146: PORT STATE SERVICE 6969/tcp closed acmsoda
...
appears the tracker is down.
This is now fixed ... the provider of that server noticed it was serving a lot of traffic {ummm ... we just did 2 point releases ... no kidding :) }, so they decided to reboot it for me :(
That did shutdown the tracker, but it is back on-line now.
Thanks, Johnny Hughes
On Wed, 2007-12-19 at 04:35 -0600, Johnny Hughes wrote:
John R Pierce wrote:
William L. Maltby wrote:
On Tue, 2007-12-18 at 15:46 -0800, John R Pierce wrote:
William L. Maltby wrote:
<snip>
# nmap -sT -p6969 -vv torrent.centos.org ... Interesting ports on 66.147.238.146: PORT STATE SERVICE 6969/tcp closed acmsoda
...
appears the tracker is down.
This is now fixed ... the provider of that server noticed it was serving a lot of traffic {ummm ... we just did 2 point releases ... no kidding :) }, so they decided to reboot it for me :(
That did shutdown the tracker, but it is back on-line now.
Going gangbusters ATM. Is it possible to have multiple trackers available? If so, sounds like it would save some of us users work occasionally. But I don't know if it adds substantially to the CentOS team workload.
Thanks, Johnny Hughes
<snip sig stuff>
Thx for this and all that y'all do!
--On Wednesday, December 19, 2007 6:44 AM -0500 "William L. Maltby" CentOS4Bill@triad.rr.com wrote:
Is it possible to have multiple trackers available?
Several clients know how to do that.
Johnny Hughes wrote:
That did shutdown the tracker, but it is back on-line now.
FWIW, the tracker is offline again tonight.
I'm seeing a few 100 DHT peers for each CentOS-5.1-{i386|x86_64}-bin-DVD so the torrent is freely running, but the tracker has been AWOL for hours now ('offline - timed out').
On Sun, 2007-12-30 at 03:36 -0800, John R Pierce wrote:
Johnny Hughes wrote:
That did shutdown the tracker, but it is back on-line now.
FWIW, the tracker is offline again tonight.
I'm seeing a few 100 DHT peers for each CentOS-5.1-{i386|x86_64}-bin-DVD so the torrent is freely running, but the tracker has been AWOL for hours now ('offline - timed out').
FYI, FWIW, still down @ 09:38 EST.
<snip>
William L. Maltby wrote:
On Sun, 2007-12-30 at 03:36 -0800, John R Pierce wrote:
Johnny Hughes wrote:
That did shutdown the tracker, but it is back on-line now.
FWIW, the tracker is offline again tonight.
I'm seeing a few 100 DHT peers for each CentOS-5.1-{i386|x86_64}-bin-DVD so the torrent is freely running, but the tracker has been AWOL for hours now ('offline - timed out').
FYI, FWIW, still down @ 09:38 EST.
<snip>
Thanks guys ... this server is down :(
trying to get hold of the provider now.
as I said prematurely, it appears the tracker is down, HOWEVER... my uTorrent (MS Windows based) client is nicely finding 107 peers using distributed tracking techniques (which I don't fully understand)
I'm getting wire speeds just a few minutes after connecting.
On Tue, 2007-12-18 at 16:12 -0800, John R Pierce wrote:
as I said prematurely, it appears the tracker is down, HOWEVER... my uTorrent (MS Windows based) client is nicely finding 107 peers using distributed tracking techniques (which I don't fully understand)
I'm getting wire speeds just a few minutes after connecting.
Earlier, I tried using the rtorrent "enable_trackers=no" setting, but the man page says its useful for use with the scheduler. It had no effect on my attempts.
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1? ATM, trying what I can recognize in the rtorrent man page, I've not seen anything that my knowledge allows me to interpret as being able to run so far.
<snip sig stuff>
Off to eat, back shortly.
Thanks,
William L. Maltby wrote:
On Tue, 2007-12-18 at 16:12 -0800, John R Pierce wrote:
as I said prematurely, it appears the tracker is down, HOWEVER... my uTorrent (MS Windows based) client is nicely finding 107 peers using distributed tracking techniques (which I don't fully understand)
I'm getting wire speeds just a few minutes after connecting.
Earlier, I tried using the rtorrent "enable_trackers=no" setting, but the man page says its useful for use with the scheduler. It had no effect on my attempts.
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1? ATM, trying what I can recognize in the rtorrent man page, I've not seen anything that my knowledge allows me to interpret as being able to run so far.
I believe azureus supports distributed tracking, as this is a java app, it should work on centos
otherwise, dunno what to suggest.
William L. Maltby wrote on Tue, 18 Dec 2007 19:37:17 -0500:
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1?
It's called DHT in utorrent and is indeed working quite nice without a tracker. Maybe there are Linux clients that support DHT?
Kai
as mentioned before there is azureus
http://azureus.sourceforge.net/
runs on java though but supports dht
mike
On 12/19/07, Kai Schaetzl maillists@conactive.com wrote:
William L. Maltby wrote on Tue, 18 Dec 2007 19:37:17 -0500:
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1?
It's called DHT in utorrent and is indeed working quite nice without a tracker. Maybe there are Linux clients that support DHT?
Kai
-- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Wed, 2007-12-19 at 13:06 +0100, Kai Schaetzl wrote:
William L. Maltby wrote on Tue, 18 Dec 2007 19:37:17 -0500:
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1?
It's called DHT in utorrent and is indeed working quite nice without a tracker. Maybe there are Linux clients that support DHT?
Kai
Thanks to both Kai and Michael Simpson for the clues to get my search started. I yum installed, from rpmforge (big thanks to Dag), bittorrent- gui.noarch (4.4.x) and went to
file:///usr/share/doc/bittorrent-4.4.0/TRACKERLESS.txt
Looks like it might be the ticket. Seems to be supporting what y'all called DHT. It says it will not mess with tracked downloads. It supports conventional tracked torrents too.
I'm going to give it a try and will report back.
Again, thanks to all for the help, including Florin, John and Johnny for the help in initial problem resolution.
Thx,
--On Wednesday, December 19, 2007 12:37 PM -0500 "William L. Maltby" CentOS4Bill@triad.rr.com wrote:
file:///usr/share/doc/bittorrent-4.4.0/TRACKERLESS.txtLooks like it might be the ticket. Seems to be supporting what y'all called DHT. It says it will not mess with tracked downloads. It supports conventional tracked torrents too.
Perhaps the next CentOS torrents can add the appropriate records to take advantage of this?
Kenneth Porter wrote on Wed, 19 Dec 2007 14:08:56 -0800:
Perhaps the next CentOS torrents can add the appropriate records to take advantage of this?
AFAIK, there is nothing special about it, nothing to add. A DHT aware client just connects to other DHT clients and these again other clients to find those clients that have the torrent or parts of it available. It just works - if there are clients that already have the file and got it with the same torrent file, so the hash matches.
Kai
On Thu, 2007-12-20 at 00:32 +0100, Kai Schaetzl wrote:
Kenneth Porter wrote on Wed, 19 Dec 2007 14:08:56 -0800:
Perhaps the next CentOS torrents can add the appropriate records to take advantage of this?
AFAIK, there is nothing special about it, nothing to add. A DHT aware client just connects to other DHT clients and these again other clients to find those clients that have the torrent or parts of it available. It just works - if there are clients that already have the file and got it with the same torrent file, so the hash matches.
Kai
Ah, yes. It sounds so simple. But I've been perusing the various references available (bittorrent ones are sparse - no man pages) and if I want to seed beginning with the ones I've already downloaded, it seems to get more complicated. Port forwarding through my firewall, generation and publishing (I currently have no http presence) of a torrent file, getting one of my computers to the DMZ (I use IPCop and right now all are in the green zone),... In fact, the /usr/share/doc bitorrent files say I need to start with a tracker.
I presume this assumes I'm downloading originally. Then, if starting in tracking mode, how does one switch to "trackerless" or DHT? From reading some of the refs in other posts, it seems that I would need to be downloading from a client that supports the DHT schema. Meaning that when I start downloading, my presence is added to the hash (or is it routing?) tables and forwarded to peers in the network and I would have to receive them also. Does the service CentOS is using support all this? I don't know.
William L. Maltby wrote:
Ah, yes. It sounds so simple. But I've been perusing the various references available (bittorrent ones are sparse - no man pages) and if I want to seed beginning with the ones I've already downloaded, it seems to get more complicated. Port forwarding through my firewall, generation and publishing (I currently have no http presence) of a torrent file, getting one of my computers to the DMZ (I use IPCop and right now all are in the green zone),... In fact, the /usr/share/doc bitorrent files say I need to start with a tracker.
when the centos tracker was down, I fired up uTorrent on a Windows box w/ the 5.1 i386 dvd torrent, and it managed to find a few dozen peers in a few minutes and pretty quickly was downloading at close to my wirespeed. Within about half an hour, I was uploading at 60% of my bandwidth and still climbing, and it showed that there were 118 or something available peers, but uTorrent tends to only connect to 30 or so at once to keep the traffic efficient.
i just fired it back up to go ahead and share, i'm seeing 180 seeds and 190 peers on i386, and 116/54 on x86_64... i'll leave it running for the holidays at least.
On Wed, 2007-12-19 at 18:20 -0800, John R Pierce wrote:
William L. Maltby wrote:
Ah, yes. It sounds so simple. But I've been perusing the various references available (bittorrent ones are sparse - no man pages) and if I want to seed beginning with the ones I've already downloaded, it seems to get more complicated. Port forwarding through my firewall, generation and publishing (I currently have no http presence) of a torrent file, getting one of my computers to the DMZ (I use IPCop and right now all are in the green zone),... In fact, the /usr/share/doc bitorrent files say I need to start with a tracker.
when the CentOS tracker was down, I fired up uTorrent on a Windows box w/ the 5.1 i386 dvd torrent, and it managed to find a few dozen peers in a few minutes and pretty quickly was downloading at close to my wirespeed. Within about half an hour, I was uploading at 60% of my bandwidth and still climbing, and it showed that there were 118 or something available peers, but uTorrent tends to only connect to 30 or so at once to keep the traffic efficient.
Still blissful (ignorance is ...), hit me with a clue bat if I'm way off base here.
Now, that makes sense (IIUC the implications of all I've read and what y'all have posted). You had already been successfully connected and acquired the DHT during previous sessions. I can guess that this would be used in place of an (un?)available tracker.
However, if one had not used a DHT-enabled client previously (my case), the CentOS torrent server was inoperative and the torrent file provided only specified one tracker URL, one should be SOL.
After examining some other torrent files, it *seems* that multiple tracker URLs can be specified. Maybe this is all it takes to eliminate brief outages such as were recently experienced by non-DHT-enabled clients?
This means that some community member would need to host a redundant tracker, torrents and images and the torrent file at the various trackers would have a complete list.
Is it worth the effort? AFAICR, the recent outage was a wetware problem at a "critical" time and, ironically, caused by the bad judgment when the resulting workload was seen.
I don't recall other instances of unavailability. Is 99.9(...9?) good enough? I guess it depends which end of the pipe you're on and attitude ATM ;-)
i just fired it back up to go ahead and share, I'm seeing 180 seeds and 190 peers on i386, and 116/54 on x86_64... I'll leave it running for the holidays at least.
I killed the rtorrent for the two 5.1 torrents and started bittorrent- GUI on them. All is working well and the rtorrent, delivering 4.6 stuff, and bittorrent (5.1 stuff) are playing nicely and sharing the bandwidth well.
After seeding for an indeterminate time, I'll kill the GUI and start the console or ncurses bittorrent with the --trackerless option and see if it uses the DHT/routing and other information that bittorrent seems to stash in ~/.bittorrent/data directory. This depends on the presence of other active clients, of course.
I'll post backin a day or two.
Meanwhile, redundancy anyone?
<snip sif stuff>
when the CentOS tracker was down, I fired up uTorrent on a Windows box w/ the 5.1 i386 dvd torrent, and it managed to find a few dozen peers in a few minutes...
Still blissful (ignorance is ...), hit me with a clue bat if I'm way off base here.
Now, that makes sense (IIUC the implications of all I've read and what y'all have posted). You had already been successfully connected and acquired the DHT during previous sessions. I can guess that this would be used in place of an (un?)available tracker.
last thing I'd torrented off centos was 5.0 prior to this... Not sure that would have any impact on hooking up with DHT on a different torrent (but, I freely admit, I haven't the vaguest clue how the DHT actually hooks up and finds the initial peers)
On Thu, 2007-12-20 at 10:34 -0800, John R Pierce wrote:
when the CentOS tracker was down, I fired up uTorrent on a Windows box w/ the 5.1 i386 dvd torrent, and it managed to find a few dozen peers in a few minutes...
Still blissful (ignorance is ...), hit me with a clue bat if I'm way off base here.
Now, that makes sense (IIUC the implications of all I've read and what y'all have posted). You had already been successfully connected and acquired the DHT during previous sessions. I can guess that this would be used in place of an (un?)available tracker.
last thing I'd torrented off centos was 5.0 prior to this... Not sure that would have any impact on hooking up with DHT on a different torrent (but, I freely admit, I haven't the vaguest clue how the DHT actually hooks up and finds the initial peers)
<supposition mode from past life> IIRC and IIUC, the hashed keys you acquired, and http addresses that were saved (if it's like bittorrent does it) could get it going. My reading of the docs indicates that the hashed keys identify torrents and the keys define name spaces, which are further sub-divided and associated with nodes in the swarm to accomplish routing optimization. Connecting with any peer with any key should cause you to receive a new routing table, which I presume could (but may not need to) contain any/all keys (torrent info?) that any node in the swarm had seen. Since the hash algorithm that "defines" the CentOS images is well-known and common, that could provide an entry even when there was no tracker if the existing IP addresses (again, I assume they are retained) are polled and any of them are running a DHT-enabled client and you possess a torrent file with the appropriate key. <supposition mode from past life>
Regardless of my meanderings, I'm comfortable now that "It Just Works". I just can't stand not knowing how once I get curious. Hangover from a past life.
<snip sig stuff>
I'll post to one of the threads again if/when I complete my attempt at making an RPM for the later bittorrent releases.
To the list: apologies for how for OT this has gone, but I've really benefited and enjoyed it all. Maybe the stupid politics isn't really enough to keep me from re-entering the biz again after all.
8-O <*slaps face*>
On Thu, 2007-12-20 at 07:42 -0500, William L. Maltby wrote:
On Wed, 2007-12-19 at 18:20 -0800, John R Pierce wrote:
William L. Maltby wrote:
Ah, yes. It sounds so simple. <snip> ... In fact, the /usr/share/doc bitorrent files say I need to start with a tracker.
when the CentOS tracker was down, I fired up uTorrent on a Windows box w/ the 5.1 i386 dvd torrent, and it managed to find a few dozen peers in a few minutes and pretty quickly was downloading at close to my wirespeed. Within about half an hour, I was uploading at 60% of my bandwidth and still climbing, and it showed that there were 118 or something available peers, but uTorrent tends to only connect to 30 or so at once to keep the traffic efficient.
Still blissful (ignorance is ...), hit me with a clue bat if I'm way off base here.
Now, that makes sense (IIUC the implications of all I've read and what y'all have posted). You had already been successfully connected and acquired the DHT during previous sessions. I can guess that this would be used in place of an (un?)available tracker.
<snip a bunch of guesses and theorizing>
i just fired it back up to go ahead and share, I'm seeing 180 seeds and 190 peers on i386, and 116/54 on x86_64... I'll leave it running for the holidays at least.
I killed the rtorrent for the two 5.1 torrents and started bittorrent- GUI on them. All is working well and the rtorrent, delivering 4.6 stuff, and bittorrent (5.1 stuff) are playing nicely and sharing the bandwidth well.
After seeding for an indeterminate time, I'll kill the GUI and start the console or ncurses bittorrent with the --trackerless option and see if it uses the DHT/routing and other information that bittorrent seems to stash in ~/.bittorrent/data directory. This depends on the presence of other active clients, of course.
I'll post backin a day or two.
No go. See my reply, later today, to
Re: [CentOS] Trackerless torrents (was: Can't connect to torrent tracker)
Briefly, with the bittorrent from Rpmforge, which is old, a .torrent file that specifies a tracker cause the bittorrent to fail if the tracker can not be contacted.
That other reply includes a compressed "log" of activities and I would be interested what results other clients give when the same test is emulated.
Meanwhile, redundancy anyone?
<snip>
I'll do an abbreviated emulation of these tests with rtorrent later, but I expect similar results.
1198111906.5525.38.camel@centos01.homegroannetworking> X-Rcpt-To: centos@centos.org
William L. Maltby wrote on Wed, 19 Dec 2007 19:51:39 -0500:
Ah, yes. It sounds so simple. But I've been perusing the various references available (bittorrent ones are sparse - no man pages) and if I want to seed beginning with the ones I've already downloaded, it seems to get more complicated.
I think you are trying to seed a *new* "unofficial" torrent. That's not what you want to do. See below.
I presume this assumes I'm downloading originally. Then, if starting in tracking mode, how does one switch to "trackerless" or DHT? From reading some of the refs in other posts, it seems that I would need to be downloading from a client that supports the DHT schema. Meaning that when I start downloading, my presence is added to the hash (or is it routing?) tables and forwarded to peers in the network and I would have to receive them also. Does the service CentOS is using support all this?
It doesn't need to support this. The important thing is that *you* have to use a DHT client. utorrent uses this automatically (you can switch it on/off in the options, though). So, you have to look in the client documentation if and how it supports DHT (which seems to be the abbreviation for Distributed Hash Table). What you then simply do is fire up that client and get the CentOS torrent and keep it running once you got it. That's all. In case there's no tracker connectable DHT-aware clients will talk to each other and look for the hash. Which means if you already got it you can serve it. Or if you don't have it yet you can get it from others. Without a tracker. The important part is that all the DHT clients use the same torrent file for starting up the download. If you download the file by other means and then start seeding it (by creating your own torrent file and uploading it to a tracker) you start a new torrent "cloud" or whatever you want to call it. I'm sure there are ways to "trick" around this and use an existing torrent file with data that were downloaded a different way. Maybe just start a download, stop it and then replace the file with a complete one.
Here's what's currently getting seeded: http://www.mininova.org/search/?search=Centos+5.1 The seed/leech ratio seems to be ok.
Kai
On Thu, 2007-12-20 at 17:43 +0100, Kai Schaetzl wrote:
1198111906.5525.38.camel@centos01.homegroannetworking> X-Rcpt-To: centos@centos.org
William L. Maltby wrote on Wed, 19 Dec 2007 19:51:39 -0500:
Ah, yes. It sounds so simple. But I've been perusing the various references available (bittorrent ones are sparse - no man pages) and if I want to seed beginning with the ones I've already downloaded, it seems to get more complicated.
I think you are trying to seed a *new* "unofficial" torrent. That's not what you want to do. See below.
Yep, it was what I wanted to do then either! :-{ But in my learning curve, thinking I needed a local torrent to get started ...
Anyway, it didn't take long to realize the implications of that and stop it.
I presume this assumes I'm downloading originally. Then, if starting in tracking mode, how does one switch to "trackerless" or DHT? From reading some of the refs in other posts, it seems that I would need to be downloading from a client that supports the DHT schema. Meaning that when I start downloading, my presence is added to the hash (or is it routing?) tables and forwarded to peers in the network and I would have to receive them also. Does the service CentOS is using support all this?
It doesn't need to support this. The important thing is that *you* have to use a DHT client. utorrent uses this automatically (you can switch it on/off in the options, though). So, you have to look in the client documentation if and how it supports DHT (which seems to be the abbreviation for Distributed Hash Table). What you then simply do is fire up that client and get the CentOS torrent and keep it running once you got it. That's all. In case there's no tracker connectable DHT-aware clients will talk to each other and look for the hash. Which means if you already got it you can serve it. Or if you don't have it yet you can get it from others. Without a tracker. The important part is that all the DHT clients use the same torrent file for starting up the download. If you download the file by other means and then start seeding it (by creating your own torrent file and uploading it to a tracker) you start a new torrent "cloud" or whatever you want to call it. I'm sure there are ways to "trick" around this and use an existing torrent file with data that were downloaded a different way. Maybe just start a download, stop it and then replace the file with a complete one.
Through experimentation, I found it's even better than that: no trickery needed!
With bittorrent, I simply positioned myself in the correct directory, configured things, and fired up the torrents (genuine CentOS ones). Even though the stuff was downloaded via conventional http methods, the images and *sum files were seen by bittorrent, the checksums were run and the images began immediately acting as seeds. Lots of peers came on board and began downloading from these images. No downloads to me were attempted. Last run of
strings ~/.bittorrent/data/routing_table | wc -l
shows 188 IP addresses. Some from the -curses and some from the GUI, I suspect. I wonder if they are stepping on each other here? No matter for now.
When I fired up the -curses version, I specified -- start_trackerless_client (IIRC, thats how it's spelled - bash_history will remember for me) and it worked fine.
I just need to find a way to make sure that it can't find a conventional tracker to see if that parameter has the desired effect. From what you say, and what I think I've learned, should work OK. And may not even be needed.
I note that there are 5.2.* and 6-beta versions of bittorrent available. I though I might see if the prerequisites for them are on my 5.1 CentOS (that my "experimental" node) and take my first shot at making an RPM to contribute back to the community (rpmforge seems to be the appropriate target).
Here's what's currently getting seeded: http://www.mininova.org/search/?search=Centos+5.1 The seed/leech ratio seems to be ok.
I book-marked that. Thanks.
Kai
Thanks to you and all who helped me get started on this.
--On Thursday, December 20, 2007 2:02 PM -0500 "William L. Maltby" CentOS4Bill@triad.rr.com wrote:
I just need to find a way to make sure that it can't find a conventional tracker to see if that parameter has the desired effect. From what you say, and what I think I've learned, should work OK. And may not even be needed.
iptables -A OUTPUT -d torrent.centos.org -p tcp --dport 6969 -j DROP
On Thu, 2007-12-20 at 13:58 -0800, Kenneth Porter wrote:
--On Thursday, December 20, 2007 2:02 PM -0500 "William L. Maltby" CentOS4Bill@triad.rr.com wrote:
I just need to find a way to make sure that it can't find a conventional tracker to see if that parameter has the desired effect. From what you say, and what I think I've learned, should work OK. And may not even be needed.
iptables -A OUTPUT -d torrent.centos.org -p tcp --dport 6969 -j DROP
Thanks Kenneth. IIRC, I can use the IP to avoid DNS resolution and do it faster? Yep just did "man ..." and see that.
<snip sig stuff>
On Thursday, December 20, 2007 5:30 PM -0500 "William L. Maltby" CentOS4Bill@triad.rr.com wrote:
iptables -A OUTPUT -d torrent.centos.org -p tcp --dport 6969 -j DROP
Thanks Kenneth. IIRC, I can use the IP to avoid DNS resolution and do it faster? Yep just did "man ..." and see that.
The iptables command stores the resolved IP in the kernel. So the DNS lookup is done once when you install the rule, not each time a packet is passed through the rule.
If you read the rules back out with "iptables -L -n" or iptables-save, you'll see the raw IP.
On Fri, 2007-12-21 at 13:03 -0800, Kenneth Porter wrote:
On Thursday, December 20, 2007 5:30 PM -0500 "William L. Maltby" CentOS4Bill@triad.rr.com wrote:
iptables -A OUTPUT -d torrent.centos.org -p tcp --dport 6969 -j DROP
Thanks Kenneth. IIRC, I can use the IP to avoid DNS resolution and do it faster? Yep just did "man ..." and see that.
The iptables command stores the resolved IP in the kernel. So the DNS lookup is done once when you install the rule, not each time a packet is passed through the rule.
If you read the rules back out with "iptables -L -n" or iptables-save, you'll see the raw IP.
Yeah. As normal, *after* I posted I remembered that from some very early and brief work with it (or was it ipchains?). I also remembered how to delete a specific rule (or was that in ipchains too?).
Anyway, I got the needed pointers. After I do some personal stuff this weekend, I plan to hit it.
<snip sig stuff>
William L. Maltby wrote on Thu, 20 Dec 2007 14:02:47 -0500:
I just need to find a way to make sure that it can't find a conventional tracker to see if that parameter has the desired effect.
But you should allow it to contact a tracker once you are done with the tests. I think with a tracker other clients will be able to find you much quicker.
Kai
On Fri, 2007-12-21 at 17:32 +0100, Kai Schaetzl wrote:
William L. Maltby wrote on Thu, 20 Dec 2007 14:02:47 -0500:
I just need to find a way to make sure that it can't find a conventional tracker to see if that parameter has the desired effect.
But you should allow it to contact a tracker once you are done with the tests. I think with a tracker other clients will be able to find you much quicker.
That was my plan. This was just for testing to educate myself further.
Kai
On Thu, 2007-12-20 at 00:32 +0100, Kai Schaetzl wrote:
Kenneth Porter wrote on Wed, 19 Dec 2007 14:08:56 -0800:
Perhaps the next CentOS torrents can add the appropriate records to take advantage of this?
AFAIK, there is nothing special about it, nothing to add. A DHT aware client just connects to other DHT clients and these again other clients to find those clients that have the torrent or parts of it available. It just works - if there are clients that already have the file and got it with the same torrent file, so the hash matches.
AFAICT now, at lease an initial successful connection with a tracker, some time previously or currently, or multiple trackers specified in the torrent file is needed to initialize everything if the primary (only?) tracker is unavailable.
So the "special" requirements seem to be at least one successful tracker-begun session by a DHT-enabled client or multiple trackers specified in the torrent file to allow "fail over" when the primary server is down and a DHT-enabled client is making its first attempt.
I'll test (if I can) this in the next couple of days if I don't get confirmation elsewhere.
Kai
1198155557.5525.76.camel@centos01.homegroannetworking> X-Rcpt-To: centos@centos.org
William L. Maltby wrote on Thu, 20 Dec 2007 07:59:17 -0500:
AFAICT now, at lease an initial successful connection with a tracker,
You mean for you own client? No. I have successfully downloaded files with DHT in the past that weren't available via tracker anymore or the tracker didn't respond (and where the P2P sites showed "0/0 seeders/leechers). The file just needs to have "some" distribution, so it eventually gets found in the DHT client cloud.
If you mean that in general, yes, there have to be a few clients initially that can connect to a tracker because finding that one client that starts the seeding without a tracker may take a long time or fail.
Kai
On Thu, 2007-12-20 at 19:31 +0100, Kai Schaetzl wrote:
1198155557.5525.76.camel@centos01.homegroannetworking> X-Rcpt-To: centos@centos.org
William L. Maltby wrote on Thu, 20 Dec 2007 07:59:17 -0500:
AFAICT now, at lease an initial successful connection with a tracker,
You mean for you own client? No. I have successfully downloaded files with DHT in the past that weren't available via tracker anymore or the tracker didn't respond (and where the P2P sites showed "0/0 seeders/leechers). The file just needs to have "some" distribution, so it eventually gets found in the DHT client cloud.
If you mean that in general, yes, there have to be a few clients initially that can connect to a tracker because finding that one client that starts the seeding without a tracker may take a long time or fail.
Kai
I've reached the same conclusion. See another of my later posts.
On Thu, 2007-12-20 at 07:59 -0500, William L. Maltby wrote:
On Thu, 2007-12-20 at 00:32 +0100, Kai Schaetzl wrote:
Kenneth Porter wrote on Wed, 19 Dec 2007 14:08:56 -0800:
Perhaps the next CentOS torrents can add the appropriate records to take advantage of this?
AFAIK, there is nothing special about it, nothing to add. A DHT aware client just connects to other DHT clients and these again other clients to find those clients that have the torrent or parts of it available. It just works - if there are clients that already have the file and got it with the same torrent file, so the hash matches.
Given: the bittorrent client available from rpmforge is old (5.2 stable are out there and 6.0-beta, but it seems both need porting to RH/CentOS before we can test/use them). I don't know if it is DHT-enabled. I've subscribed to the Rpmforge mailing list to see if there is a way I can help make the newer ones available. It'll be slow going because many of the prerequisites are not (transparently) met. It does look as if there are some alternative implementations of the needed GTK, wx, twisted, ... items. But I know nothing about them and can't tell if that is true. I think Dag and Co. will be able to point me in the right direction.
Conclusion: current bittorrent from Rpmforge will not operate successfully on a tracked torrent without access to the tracker, AFAICT.
Activity log attached.
If others would like to emulate this test with other clients/OSs, I would find it interesting.
AFAICT now, at lease an initial successful connection with a tracker, some time previously or currently, or multiple trackers specified in the torrent file is needed to initialize everything if the primary (only?) tracker is unavailable.
If one emulates the activities detailed in the log, with modifications toward testing if any of the clients connect when no record of a tracker ever being seen exists, I would find the results interesting. Further, one could also test when a tracker has previously been seen, but is currently unavailable. I would also find that of interest.
So the "special" requirements seem to be at least one successful tracker-begun session by a DHT-enabled client or multiple trackers specified in the torrent file to allow "fail over" when the primary server is down and a DHT-enabled client is making its first attempt.
Anyone want/able to test this supposition if they find they have a client that will operate when the tracker is not available?
<snip>
Finally, it *seems* to me that a torrent must be somehow specified as trackerless for bittorrent (and others?) to handle a tracked torrent when no tracker is available. This is a supposition by me from various non-extensive readings of things during this test.
On Wed, 2007-12-19 at 12:37 -0500, William L. Maltby wrote:
On Wed, 2007-12-19 at 13:06 +0100, Kai Schaetzl wrote:
William L. Maltby wrote on Tue, 18 Dec 2007 19:37:17 -0500:
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1?
It's called DHT in utorrent and is indeed working quite nice without a tracker. Maybe there are Linux clients that support DHT?
Kai
Thanks to both Kai and Michael Simpson for the clues to get my search started. I yum installed, from rpmforge (big thanks to Dag), bittorrent- gui.noarch (4.4.x) and went to
file:///usr/share/doc/bittorrent-4.4.0/TRACKERLESS.txtLooks like it might be the ticket. Seems to be supporting what y'all called DHT. It says it will not mess with tracked downloads. It supports conventional tracked torrents too.
I'm going to give it a try and will report back.
Again, thanks to all for the help, including Florin, John and Johnny for the help in initial problem resolution.
And Jim and ... all
FYI: bittorrent-gui from rpmforge is working well on my 5.1 CentOS. My only complaint is that it seems unfriendly to the GUI-impaired, such as myself. "MORE DATA", I keep screaming. But that's the nature of GUI and me.
I see no way to test it without a tracker unavailable unless the CentOS tracker dies again.
However, the -console and -curses versions seem to support trackerless operation. If they use the data stashed in ~/.bittorrent/data directory, I should be able to use the --start_trackerless_client and see what happens. However, the --help output says this is to download trackerless torrents. What this implies, I am unsure of. Can the CentOS stuff be "trackerless" once I've acquired routing/DHT tables?
Thx,
On Thu, 2007-12-20 at 08:29 -0500, William L. Maltby wrote:
On Wed, 2007-12-19 at 12:37 -0500, William L. Maltby wrote:
On Wed, 2007-12-19 at 13:06 +0100, Kai Schaetzl wrote:
William L. Maltby wrote on Tue, 18 Dec 2007 19:37:17 -0500:
<snip>
FYI: bittorrent-gui from rpmforge is working well on my 5.1 CentOS. My only complaint is that it seems unfriendly to the GUI-impaired, such as myself. "MORE DATA", I keep screaming. But that's the nature of GUI and me.
BTW, it really does provide more data than rtorrent, but I started it assuming that it would provide less. I like the additional info provided on the "peer" screens.
<snip>
On Thu, 2007-12-20 at 08:29 -0500, William L. Maltby wrote:
On Wed, 2007-12-19 at 12:37 -0500, William L. Maltby wrote:
On Wed, 2007-12-19 at 13:06 +0100, Kai Schaetzl wrote:
William L. Maltby wrote on Tue, 18 Dec 2007 19:37:17 -0500:
Anybody know how I can do the equivalent "distributed" for CentOS on 4.6 or 5.1?
It's called DHT in utorrent and is indeed working quite nice without a tracker. Maybe there are Linux clients that support DHT?
Kai
Thanks to both Kai and Michael Simpson for the clues to get my search started. I yum installed, from rpmforge (big thanks to Dag), bittorrent- gui.noarch (4.4.x) and went to
file:///usr/share/doc/bittorrent-4.4.0/TRACKERLESS.txtLooks like it might be the ticket. Seems to be supporting what y'all called DHT. It says it will not mess with tracked downloads. It supports conventional tracked torrents too.
I'm going to give it a try and will report back.
Again, thanks to all for the help, including Florin, John and Johnny for the help in initial problem resolution.
<snip>
I see no way to test it without a tracker unavailable unless the CentOS tracker dies again.
Folks told me how.
However, the -console and -curses versions seem to support trackerless operation. If they use the data stashed in ~/.bittorrent/data directory, I should be able to use the --start_trackerless_client and see what happens. However, the --help output says this is to download trackerless torrents. What this implies, I am unsure of. Can the CentOS stuff be "trackerless" once I've acquired routing/DHT tables?
The bittorrent version from rpmforge will not do the tracked CentOS torrent if a tracker is not available. See my reply to
Re: [CentOS] Trackerless torrents (was: Can't connect to torrent tracker)
The tests outlined in the activity log attached there could be emulated with other clients to see what they do, if one is interested.
Thx,
On Tue, 2007-12-18 at 15:06 -0800, Florin Andrei wrote:
William L. Maltby wrote:
Well, yesterday my 5.1 and 4.6 systems could connect to
torrent.centos.org
OK. Today neither system can connect. I can ping, but that's all. No changes were made to the firewall during this time.
Any tips on debugging this?
Is Comcast your Internet provider?
P.S. I can also wget, which I tried after the port scan showed that was an open port.
--On Tuesday, December 18, 2007 3:06 PM -0800 Florin Andrei florin@andrei.myip.org wrote:
Is Comcast your Internet provider?
Not on the colocated host I'm connecting from. "telnet torrent.centos.org 6969" fails as well:
Trying 66.147.238.146... telnet: connect to address 66.147.238.146: Connection refused telnet: Unable to connect to remote host: Connection refused
iptables is "off" (ie. empty rule lists and ACCEPT on all tables) so any firewalling must be outside my host.
I also get "Connection refused" from my system on Speakeasy.
-
Florin Andrei -
John R Pierce -
Johnny Hughes -
Kai Schaetzl -
Kenneth Porter -
Michael Simpson -
William L. Maltby