opencaching.com Geocaching by the community, for the community italian charms 87744    FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  Distributed Data Network Goto page Previous  1, 2, 3, 4, 5, 6          opencaching.com Forum Index -> Tech Stuff View previous topic :: View next topic   Author Message marc Joined: 01 Oct 2003 Posts: 61 Posted: Sat Oct 18, 2003 3:22 pm    Post subject: All, stats update: kdh and me are testing rsync with hundret thousand of single files. Looks good until now - and i will post our results in 1-2 days! Marc. Back to top Team BMW-Biker Joined: 01 Oct 2003 Posts: 209 Posted: Sat Oct 18, 2003 3:49 pm    Post subject: Hi, Quote: There are a lot of reasons to archive a cache. For all of them the cache should archived everywhere it's listed. Most caches that are archived are part of the history of the sport and the logs and caches should be available to read. Archived means nothing more than the cache is no longer available to find, but it is available to look at for posterity. yes, we've thougt the same thing ... Quote: While this problem should not happen often we need to be able to delete the cache on all nodes, even if a node owner doesn't want too. 1. (core)node owner are "trusted" people (see security thread) 2. if a corenode-owner doesnt want to accept changes/deletes from other corenodes, we cannot trust him any more and he will be no more "official opencaching-network corenode". 3. informations which are syndicated one time are available "anywhere else", sure we can Quote: go back to the Landowner and say "it's gone" but its only be gone from the "official corenodes" (i think everybody of us knows that) Quote: then we have no credibility and we are not meeting one of the missions of this organizatoin. ok, i think this view is a little bit US-specific (maybe this will come in germany the same way - when caching comes more public) But you are right - we should be able to remove cache/log/picture informations from all official corenodes. In my "humidity" opinion it would be the easiest to set a text like "Removed because ...", take the "phsical" cache away and give the owner no more (write) access to it (in special cases deactivating the user-account ... even this has no effect because this "critical" users could sign up on a new account). oh, this goes to the discussion of approvers or not ... ... Oli Back to top Renegade Knight Joined: 15 Sep 2003 Posts: 151 Posted: Tue Oct 21, 2003 11:54 pm    Post subject: The landowner thing probably isn't specific to the USA but, in talking with Nici- it's not a problem in Germany. From what I understand you are more likely to get a call saying "hey I found this and is it yours?" Which is a different problem we need to be able to solve. The key is to be able to solve both. Thanks for your help. I have a better idea of what is possible with the distribruted database. Back to top marc Joined: 01 Oct 2003 Posts: 61 Posted: Wed Oct 22, 2003 12:28 pm    Post subject: 1. rsync test completed! Hi, here are our rsync test results: We tested with two machines. One is a Pentium 266mhz with 100MB RAM connected via T-Online ADSL - let him call Server A - and the other is a Celeron 2Ghz with 256MB RAM connected via 2MBit SDSL which we call Server B. 1. Test - Set up a new core node. Server A has all data and syncs it to Sever B which has no data: Code: hardware (sender) = P2/266 Mhz, 100MB Ram bandwith (sender) = DSL, 198bit/s upstream = 24 kB/s data 1 = 4000 pics (avarage 60kB) = 235MB data 2 = 50.000 cachedescriptions (avarage 2kB) = 86MB avarage CPU load = 10% avarage use of bandwith = 100% total time = 215min = 3h 35min 2. Test - Server A changed 10 files Code: hardware (sender) = P2/266 Mhz, 100MB Ram bandwith (sender) = DSL, 198bit/s upstream = 24 kB/s          data 1 = 4000 pics (avarage 60kB) = 235MB data 2 = 50.000 cachedescriptions (avarage 2kB) = 86MB avarage CPU load = 100% avarage use of bandwith = 100% total time = 4min 3. Test - Server B synced 50.000 new files to Server A Code: load average: 0.00% total: matches=0  tag_hits=0  false_alarms=0 data=46534363 rsync[5053] (sender) heap statistics:   arena:       20054424   (bytes from sbrk)   ordblks:            4   (chunks not in use)   smblks:             0   hblks:              1   (chunks from mmap)   hblkhd:       1028096   (bytes from mmap)   usmblks:            0   fsmblks:            0   uordblks:    20031088   (bytes used)   fordblks:       23336   (bytes free)   keepcost:        2648   (bytes in releasable chunk) Number of files: 203198 Number of files transferred: 50015 Total file size: 399881359 bytes Total transferred file size: 46534363 bytes Literal data: 46534363 bytes Matched data: 0 bytes File list size: 3057451 Total bytes written: 51592426 Total bytes read: 800260 wrote 51592426 bytes  read 800260 bytes  16530.27 bytes/sec total size is 399881359  speedup is 7.63 real    52m49.070s user    0m17.190s sys     0m24.240s 4. Test - Server B synced 390.967 xml files to Sever A Code: total: matches=0  tag_hits=0  false_alarms=0 data=574629222 rsync[9622] (sender) heap statistics:   arena:       38359448   (bytes from sbrk)   ordblks:            6   (chunks not in use)   smblks:             0   hblks:              2   (chunks from mmap)   hblkhd:       2318336   (bytes from mmap)   usmblks:            0   fsmblks:            0   uordblks:    38234440   (bytes used)   fordblks:      125008   (bytes free)   keepcost:      100904   (bytes in releasable chunk) Number of files: 390967 Number of files transferred: 390946 Total file size: 574629451 bytes Total transferred file size: 574629222 bytes Literal data: 574629222 bytes Matched data: 0 bytes File list size: 5348780 Total bytes written: 595635778 Total bytes read: 6255198 wrote 595635778 bytes  read 6255198 bytes  43443.72 bytes/sec total size is 574629451  speedup is 0.95 rsync error: partial transfer (code 23) at main.c(578) real    230m53.917s user    2m21.120s sys     7m31.260s 5. Test - Server A changed 196.325 files out of a total of 399.531 Code: total: matches=9352  tag_hits=12118  false_alarms=7 data=182299206 rsync[25438] (sender) heap statistics:   arena:       39109016   (bytes from sbrk)   ordblks:            7   (chunks not in use)   smblks:             0   hblks:              2   (chunks from mmap)   hblkhd:       2318336   (bytes from mmap)   usmblks:            0   fsmblks:            0   uordblks:    39084272   (bytes used)   fordblks:       24744   (bytes free)   keepcost:        1352   (bytes in releasable chunk) Number of files: 399531 Number of files transferred: 196325 Total file size: 582440654 bytes Total transferred file size: 182567818 bytes Literal data: 182299206 bytes Matched data: 268612 bytes File list size: 5436645 Total bytes written: 195626263 Total bytes read: 5111602 wrote 195626263 bytes  read 5111602 bytes  22209.20 bytes/sec total size is 582440654  speedup is 2.90 rsync error: partial transfer (code 23) at main.c(578) real    150m38.245s user    3m13.860s sys     2m4.460s As you can see - its no problem for rsync to handle hundret thousand of files in a single dirrectory. I hope this helps us to decide wether or not to use rsync. The opencaching rsync daemon is ready for another test. If someone on the US side would like to test it would be great :) Marc. Back to top nici- Joined: 25 Sep 2003 Posts: 199 Location: Hürth near Cologne, Germany Posted: Wed Oct 22, 2003 4:05 pm    Post subject: Renegade Knight wrote: From what I understand you are more likely to get a call saying "hey I found this and is it yours?" Which is a different problem we need to be able to solve. True. Got a Mail only yesterday, saying 'my son found one of your treasures. What to do now?' That is _one_ thing. We were also contacted once from a local 'lost & found' that someone had brought a cache box and if we knew the owner. Another thing, that happens from time to time is, that a popular cache is lost or broken. If then the original owner is not able or willing to react soon, some other cacher (e.g. a local one) just does what has to be done to set it up again. So far we have not been contacted by any land owner or ranger or such folks. May be they just throw away a cache, if it annoyes them. But also 'missing' caches are not that frequnet over here so far and if, one can often easily see tracks of wild sows or other animals. Most 'officials' that contact us are from local tourist offices, that want to boost tourist business with geocaching. And journalists of course who want to get a mail address or phone number of local cachers from xy *sigh*. And companies who want to use geocaching as an incentive or such a thing. nici- _________________ Geocaching.de Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         opencaching.com Forum Index -> Tech Stuff All times are GMT - 5 Hours Goto page Previous  1, 2, 3, 4, 5, 6 Page 6 of 6   Jump to: Select a forum OpenCaching.com----------------General (The Catch All)Issues, decisions, answersLegal StuffTech StuffContentDesign WizardsCode MonkeysJust Say Hi Projects----------------Geo TagsHosting ScriptsGPX Standards Diskussionsforen----------------Opencaching.de Dutch Mirror----------------Opencaching.nl Opencaching The Organization----------------Let's Get Organized  Powered by phpBB 2.0.6 © 2001, 2002 phpBB Group