Poll: Tell me more about fault tolerant file storage?
You do not have permission to vote in this poll.
No as we have said before, GO AWAY!
0%
0 0%
Yeah man sounds seriously cool and cutting edge.
100.00%
2 100.00%
Total 2 vote(s) 100%
* You voted for this item. [Show Results]

Torrents VS Fault Tollerance
#11
I'm not looking for the files you are downloading, I'm looking for the torrents, or at least the hash for the torrent so I can try to download it myself and see what happens.

If I can download without error, then you'll definitely want to poke around your system to see what is causing the failures. If I do get errors, I'll have a better idea what you are experiencing.
Reply
#12
Certain versions of uTorrent have the nasty tendency to move incomplete files to a 'save' directory.
Ideally all incomplete files should have an extra extension added, such a .!ut for utorrent and !qb for qbittorrent.
If this feature is not applied, all sorts of bloody hell can arise.

I have even seen this with qbittorrent which does recheck all downloads before moving to the save directory, though fortunately it is rare.

My guesses on the issue are:
1. Client error: sometimes the software just takes a pizz.

2. Also make sure the number of downloads/uploads/connections are set to a reasonable level, and that your system and router can handle it - or some very strange things can happen.

3. System caching errors: Data in -> garbage out. Mess around with cache options, redownload and see if anything different.

4. Torrent errors. Garbage in -> garbage out. Some perfectly good posters will post some perfectly lousy crap simply because they did not fully test their offerings. Seed poisoning is also a possibility as I have often turned on Peer Blocker and noticed transfer rates go UP - suggesting there is something rotten in the swarm.

5. Malware. I would be very suspicious with anything that has titles catering to pop culture like 1000 GREATEST xxxx IN ALL TIME. Generally after a period of time the boogers IPs die and the booger starts to rot, leaving non-functional files. Check the posters, to make sure they are not sporgers, though most sporgers are deleted here after a few days or so. In the OTHER section there is some dildo posting fake movie files, for example.

6. Hard drive. Check the drive with a *good* tester such as HD Sentinel v5 which can catch and fix CRC errors that CHKDSK will often miss. I've noted (on another thread) that some drives simply do not like torrents, or possibly their particular settings. I'm still trying to ferret out the details on that issue.

Try downloading the torrents in question from another machine. Keeping in mind that indeed, there are bad torrents posted.

Some torrents are reposted due to errors found on first try. This indicates that some part of the upload functions may be somehow responsible. Or even in the archiving methods.

Also: If a torrent is 8 years old with a single seeder, it may be on an ancient 8 year old seedbox, which if unmaintained can accrue errors over time. I know ext3/4 is supposed to be errorproof, but i dont believe it.
Reply
#13
(Nov 26, 2018, 12:14 pm)Moe Wrote: I'm still calling bullshit.  The possibility of an entire swarm having the same bad data that wasn't already rejected by standard piece checks is next to impossible.

On the other hand, downloaded data being modified by another process such as an anti-virus or, in the case of music files, just listening to them, is a regular occurrence.

Share the torrent links if you have some examples of one of these bad swarms.
MEGA SAMPLES VOL-40
magnet:?xt=urn:btih:3joc7y7djq*****jqw27oujszm4xhp53fuv&dn=MEGA%20SAMPLES%20VOL-40&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80

MEGA SAMPLES VOL-71
magnet:?xt=urn:btih:cskyq7isve5btnscb62jyaa6aftbqfhp&dn=MEGA%20SAMPLES%20VOL-71&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80

MEGA SAMPLES VOL-82
magnet:?xt=urn:btih:t2zzomjcvkh3i2rcr4spuz3f4x2r3nok&dn=MEGA%20SAMPLES%20VOL-82&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80

MEGA SAMPLES VOL-59
magnet:?xt=urn:btih:4dkqcmkwgh66aimhan5k5ecm5nycg4ao&dn=MEGA%20SAMPLES%20VOL-59&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80

MEGA SAMPLES VOL-104
magnet:?xt=urn:btih:eynq6fl7lkllxfvlmfy53q2d5rozloyz&dn=MEGA%20SAMPLES%20VOL-104&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80

The last 2 i get the majority of but the first 2 are 80% corrupt the 3rd being about 33% corrupt.

Also I can't get VOL-59 to completion anymore.

I just got 104 to completion without error, maybe its the hard disk i'm using? I dunno but ignore me until I have the other torrents again.

Multi(it looks like all the individual tracks from the guitar "?heroes?" games) 250gb reset to 242gb after recheck magnet:?xt=urn:btih:7k7m4dcswio6t3rlrj3dgan3irwppw54&dn=Multi&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80

Maybe its my hdd? It did make the click of death once, but its been fine since external segate usb3 2tb
Reply
#14
Big torrents around 50GB each (size of a Rodney's movie) and poorly seeded, with my speed it would take weeks. Samples 40 (50%) and 59 (90%) are incomplete.

Problem with big collections and the use of RAR archives: They're good to pack files but one missing/bad piece and they can become useless. Collections make sense for audio distribution, but I understand few live long in the torrent wilderness.

Multi (250GB) is a lot of media files, we should use "hash-lists" like we did with playlists instead of those mega-torrents.
Reply
#15
For all we know, you may be dealing with a fake torrent. At least that's what torrentz2 states.

As mentioned by Moe, every piece is checked. If the SHA-1 hash check fails, the piece is corrupt and is dropped.
The .torrent file store the hash of every piece and is used to verify the integrity of the received pieces.



What would really help when asking questions is to post all the necessary information that others can use to verify the problem at hand.
So, if you have such info, please mention it.
Eg., You could say that you downloaded this torrent, but a certain file was not completing in your torrent client (client name and version). This will avoid long discussions. ;-)

And, some applications do write to the file itself. Eg., Calibre ebook viewer, stores metadata such as last read page in the file that is in the Calibre Library. I'd once uploaded a file to TPB from my Calibre Library and later used the viewer. Net result, a hash check on my torrent showed < 100%, as the file had been modified.
Reply
#16
Given I got the 104 complete and i wasnt before I believe my problem may have to do with a hard disk or usb3 interface, i will double check and post back here again if I can reliably fault a torrent again. I will post up specific files within next time so ya only have to try one small piece.
Reply
#17
Dont forget SATA connectors that can give intermittent and even hard sector errors, though typically they occur as soft crc errors, enough to putz up a download. Other day wife's machine started giving 'click of death'. Powered down, reseated connectors - problem solved. Data cables are usually worst - make sure they have metal clips (though it is no real guarantee). Cheap caps in a power supply might also generate some random errors with drives, and might be a problem with one machine here.
Reply
#18
VOL-40 and VOL-71 failed to start for me. No one wanted to give me the meta-data I guess.

VOL-82 I was able to complete with 0 hash failures. I even ran a force-recheck on it and it came back 100%

Something is definitely going on with your system. A rogue piece of software or hardware modifying data after you receive it.
Reply
#19
AV Software???

i only use a limited command line scanner occasionally, and turned off the M$ garbage, but imagine its possible for them to corrupt downloads. Especially if the system is regularly 'updated'.
Reply
#20
So I still force check my files and my conclusion is that its my software, maybe somthing in my system upsets it? It only happens on occaision and I cannot pin it down, my system goes for months at a time between reboots, as long as I can hold the UI working only then or other intermittent issue, i reboot. Not sure what it could be because as I have said I keep chrome open for weeks at a time, i'd use Firefox but in last test it showed heavy disk IO (both reading and writing) and thatl kill my ssd sooner. ( I still use firefox to download video offa youtube) Anyone have anything similar happen to them?

My software, Win 7x64 and Tixati
Reply




Users browsing this thread: 1 Guest(s)