 |
opencaching.com
Geocaching by the community, for the community
|
| View previous topic :: View next topic |
| Author |
Message |
ju66l3r
Joined: 18 Sep 2003
Posts: 10
|
 Posted: Fri Sep 19, 2003 5:00 am Post subject: Some Ideas on filling a cache database |
 |
|
GC.com lists all caches with an ID for the aspx page to know which cache to display when you pick a cache.
Any new solution wouldn't necessarily need to have the whole cache, just the coordinates and the link to GC.com.
With the coordinates, a whole new set of possible programming concepts comes up (directional searching, etc). Of course, there are a few ways to go about this (some kinder than others). One of the more brutal approaches is to do page-gets on ID=XXX where XXX goes from 1 to infinity...then you just parse the results. Of course, if there are 500,000 IDs then that's 500,000 hits (text only, though) in the matter of a few hours...it could easily be spread over a few days though. The next problem would be updating for new or modified caches. I'll have to think about that...
BTW, not implementing this in any way, just brainstorming. Once we have data though (even pseudo-data or something like 200 caches in a single area with a variety of types) then we can consider what it'll take to store it and how we can give it some sort of open access for everyone to program around it without killing the DB server, etc... |
|
| Back to top |
|
 |
hmarq
Site Admin
Joined: 15 Sep 2003
Posts: 351
|
| Posted: Fri Sep 19, 2003 6:33 am Post subject: |
|
|
That's a good idea. We're already discussing a the idea of an 'offsite' cache type that would allow folks the ability to host their own pages, yet still have them listed/searchable on the site. And listing GC caches, or a subset of them that are submitted by the owners is a logical extention of that.
However, while I haven't come right out and said it, I'm not planning on doing anything that gc is openly hostile toward ... we won't be sreen scraping their pages, collecting info though clandestine means, etc. When that feature is ready to use, I/we will contact the folks at geocaching.com and ask them if they want to play on that basis; if it's no, it's no and I'm ok with that, but at the same time it would seem to be consistent with some things they have hinted at in the past ... ie 'opt in' stats might be ok (which we could provide if the cache owner decides to list here, and finders play here), or the 'working on an access API' ...
We'll see where this goes, but it is a good idea. |
|
| Back to top |
|
|
ju66l3r
Joined: 18 Sep 2003
Posts: 10
|
| Posted: Fri Sep 19, 2003 1:11 pm Post subject: |
|
|
| hmarq wrote: | | That's a good idea. We're already discussing a the idea of an 'offsite' cache type that would allow folks the ability to host their own pages, yet still have them listed/searchable on the site. And listing GC caches, or a subset of them that are submitted by the owners is a logical extention of that. |
Right, but the biggest problem will be mainenance (as was mentioned in that other thread). If someone double lists and needs to edit, they'll have to do it twice if we can't find some way to d/l from GC. Nobody will want to do it twice for the sake of open source (I'm having a hard enough time just explaining what's so open about it on the GC forum).
| hmarq wrote: |
However, while I haven't come right out and said it, I'm not planning on doing anything that gc is openly hostile toward ... we won't be sreen scraping their pages, collecting info though clandestine means, etc. When that feature is ready to use, I/we will contact the folks at geocaching.com and ask them if they want to play on that basis; if it's no, it's no and I'm ok with that, but at the same time it would seem to be consistent with some things they have hinted at in the past ... ie 'opt in' stats might be ok (which we could provide if the cache owner decides to list here, and finders play here), or the 'working on an access API' ...
|
Agreed. But what I think this site and our time might be worth doing is to become and addition to GC rather than a competitor or a mirror. Since it's highly unlikely that we will get access to the entire database and nobody will want to double list (i.e. turn this into navicache v2.0), then we should build a CVS for tools to add onto geocaching.com. This might mean some reverse engineering (totally legal), but empowering people to say use a directional search function (i.e. show me all caches from my location in the NE quadrant only) by submitting their query to a page here that sends the variables into Jeremy's page and sends the person to that page (or parses it and gives them the results here). This would be no worse than if the person had gone to GC and asked for all nearby caches and filtered them by hand (but we'd have a php/C#/asp/whatever page that did the filtering for them).
I can think of only one way this would become a problem (if Jeremy added deep linking prohibition to his ToS...but actually those have not held up in most courts when they have been prosecuted as damaging).
THEN, we could have our own additional database of virtuals, locationless, caches that GC doesn't allow, that people could opt into any of the tools so they could see GC caches and OC caches in their search results, leaderboard stats, etc...
Just some ideas on how to seamlessly integrate into what we all agree is a decent product already. |
|
| Back to top |
|
|
Scout
Joined: 16 Sep 2003
Posts: 235
|
| Posted: Fri Sep 19, 2003 1:31 pm Post subject: |
|
|
| ju66l3r wrote: | | (I'm having a hard enough time just explaining what's so open about it on the GC forum). |
One difference should be that the data is freely available for geocachers to provide their own value-added services.
The other difference should be that the software developed in support of this site should be made available to the community via the GPL or something equivalent, so that geocachers can build on it when creating their own value-added services.
_________________
Scout http://GPSgames.org
 |
|
| Back to top |
|
|
hmarq
Site Admin
Joined: 15 Sep 2003
Posts: 351
|
| Posted: Fri Sep 19, 2003 1:58 pm Post subject: |
|
|
| Scout wrote: | | ju66l3r wrote: | | (I'm having a hard enough time just explaining what's so open about it on the GC forum). |
One difference should be that the data is freely available for geocachers to provide their own value-added services.
The other difference should be that the software developed in support of this site should be made available to the community via the GPL or something equivalent, so that geocachers can build on it when creating their own value-added services. |
Just for the record ... the data (less user data, like passwords, email addresses and the like) will be 100% open and distributed from the get go ... and at some point, after we actually have one, the APIs will be open ... you will not likely see an anonymous cvs server with source code to the website though ... not because you can't or shouldn't have it, but it's just an invitation to be hacked on. As the site matures and leadership cedes to a governing body ... sure there may be snapshots of the whole source released. I think you'd have a hard time finding a web developer though that wants a developing web site fully exposed before it's tightend down. |
|
| Back to top |
|
|
hmarq
Site Admin
Joined: 15 Sep 2003
Posts: 351
|
| Posted: Fri Sep 19, 2003 2:12 pm Post subject: |
|
|
| ju66l3r wrote: |
Right, but the biggest problem will be mainenance (as was mentioned in that other thread). If someone double lists and needs to edit, they'll have to do it twice if we can't find some way to d/l from GC. Nobody will want to do it twice for the sake of open source (I'm having a hard enough time just explaining what's so open about it on the GC forum).
|
I think this is easy to answer ... "it's your data, not ours", "the game should be allowed to evolve and receive support when it does", "enhancements should be user/community driven, not profit driven"
GC does a fine job with a lot of things. We're just trying to do some of the things it won't either directly, or by giving folks the data/tools to build it themselves. |
|
| Back to top |
|
|
|
Powered by phpBB 2.0.6 © 2001, 2002 phpBB Group
|
FAQ
Search 
Memberlist
Usergroups
Register
Profile 
Log in to check your private messages 
Log in 
