Some of my hobbies can either be frugal or expensive - all depends on how "into it" I want to get.
Sewing: expensive to start if you don't already have a machine. I can get patterns for 99 cents if I watch the sales at Joanns, and I can get good prices on fabric at their sales too.
Knitting and crocheting: Most all of my needles and hooks have been given to me by family, so my only real expense is yarn. I get a catalog with beautiful yarn in it (read: expensive) but so far I haven't ordered anything. Most of my yarn is the cheap acrylic or cotton stuff.
Gardening: Buying new plants can get costly, but buying small ones in 6-picks or 4-inch pots when possible, or growing from seed, makes your money go a long way. The actual gardening part is free once you have a trowel, some pruners, that sort of stuff. I try to start new plants from cuttings, and divide plants when possible, but I have to be honest - I'm not a very patient gardener.
I used to be into paper arts and rubber stampng - very pricey hobby!
The internet - DivaTribe and tinkering with web sites for my daughter (and now my dog, lol) is a hobby, and while it's not super cheap, it is starting to pay for itself.