Geocaching is basically treasure hunting with a GPS. The goal of the game is to get locations of caches from places like Geocaching.com and upload the latitude and longitude to a GPS. Then one actually has to go to the caches location and find it (it may be hidden). Inside the cache are usually trinkets that can be swapped and a log of those who have found it.
Geocaching is fun on multiple levels. First there is the fun of discovery. It truly is like "X marks the spot" treasure hunt. This fun can be increased by doing it with family or friends. Then there is the fun of actually new exploring places. When I first went geocaching I visited nearby places that I never paid attention to and even found a nice corner grocery store.
Watch out though. Some people may mistake you for really lost, weird tourists. Every geocacher has a story of being mistaken for lost, a creep, terrorist, or something similar.