Which city would be the best inexpensive place, preferably near water?

Anything AWAY from 'water' is going to be less expensive.   If you really want to 'find out'... then do some exploring.  I suggest finding a very reasonable place INLAND and somewhat central... so that you can do day-trips and over-nighters to see other areas. 

If you 'stay' at the coast... you'll always have further to travel; so staying in the 'middle' of things, will put you closer to a plethora of options.

Example:  San Ramon.   Good sized city, has ALL amenities... and of course; access to Public Transport.  They have a bus terminal where the buses go to ONLY the Airport/San Jose and Puntarenas town.  The other bus station has buses to all of the regional towns nearby, including Palmares, Naranjo, etc.  Also... there are 5 buses a day up to La Fortuna.   Around San Ramon are numerous small towns and villages with a variety of temperature zones and all of them benefit from proximity to the city and it's conveniences.

Same for Quesada... it is different, and not so 'pretty' ... but is the gateway to the north.  The bus station is huge, bi-hourly buses to airport/San Jose... and numerous buses daily to La Fortuna and all points north.  Again, Quesada has all amenities; shopping; medical; etc.  The areas around it range from high mountain towns to lower valley towns and everything inbetween.  There are tons of expats in the area... but are spread out because the options aren't limited to 'this beach' or 'that beach'.  And ... the area is agricultural and rich... so produce, dairy, and meat are more reasonable than in much of CR.

Also try San Isidro General... a southern city that is within 90 minutes to a number of south Pacific beaches.  But it is higher up, has cooler living conditions, and is close to mountains and 'nature'.

Get it???    You can rent a tiny place at the beach for $1000; or you can rent a decent sized place 'elsewhere' for $400... that will allow you to get out and see what CR really looks like.

Do some soul searching and then start a plan.   We're here to help.

Pura Vida

@rainagain wow thank you so much for your response

@Jusk

Enjoy the hunt!!   And don't just go to the 'postcard' places that are on U-Tube... the real magic is where the Ticos live.

@Jusk

Every thing in Costa Rica is expensive.  Some are just more expensive than others.  If you want less expensive stay away from all tourist areas, meaning principally Tamarindo on the West and Puerto Viejo on the East.  Stay away from Limon on the east for your safety,

If you are in the Central Valley every place you would want to locate for safety and security will be expensive.  Costa Rica is the most expensive Country in Central America.  Perhaps some of the small towns outside metro San Jose will be cheaper for rent.  Groceries maybe not.

@sporto505

I'm finding out Brazil is much cheaper to live

And Colombia is becoming very popular and way less expensive... 1.9 million expats from around the world now call Colombia home.... up 1,700% (not a typo!!) since 1990.  Lots to choose from, 14 Int'l airports; most big cities are at higher elevations, so the temps are spring or fall-like year round; etc.  Still not too far from the US and like I said, most prices are anywhere from 30-50% cheaper than Costa Rica.... including rent, groceries, transportation, etc.  Minimum SS benefits to apply for retired Residency Status is a little less than $1000 USD; but they do 'tax' your foreign income even if it is retirement.  You can always for a FTC, (foreign tax credit) when you file your taxes back in the states.   Just a thought..