I can't vouch for other types of gunge (haven't used 'em), but the most common by far is the Natrasol type.
Natrasol isn't
harmful if swallowed. In fact, it's used as a food additive (thickener and emulsifier, E461=methylcellulose, E462=ethylcellulose), and is rumoured to be among the ingredients in the McDonald's milkshake (which explains a lot). However, it's quite unpleasant to taste if it's been coloured with powder/poster paint, of course! Otherwise it's kinda... neutral, and the best way I can describe it is 'a bit like celery', but you'll want to spit it out if you get any in your mouth; it isn't nice (actually I think it's the texture rather than the taste)!
As for the effect on your stomach, it's fluid dietary fibre; hydroxyethylcellulose (or hydroxymethylcellulose, or hydroxymethylcellulose ether). That is what it
is. So, if you swallow a little, you'll be fine (though as I said above, it doesn't taste good)! If you swallow a
lot, then you may feel a bit bloated and need the toilet a couple of hours later. But you should be alright after that; it won't do you any real harm unless you have some kind of bowel condition where you should avoid fibre, though you should definitely avoid deliberately drinking the stuff (see above; it's not nice!). (If you do have a bowel condition, there's every possibility you may have been prescribed it at some point as Citrucel or whatever... oh, and it's one of the main ingredients in K-Y Jelly too...)
You really don't want to keep it once mixed up for more than about a week if you can avoid it. After two weeks, it's definitely "turned"; still sploshable, but not exactly what you'd call 100% pleasant. It's not the gunge, that keeps ages in powder form—it's the water
in it, that goes kinda... stagnant, and it starts developing a distinctive celery-esque smell. (I presume chemical preparations avoid this with distilled water or something, I don't really know for sure.)
I'm not an expert in the field, but I don't think bacteria can grow in it. It's too thick, and it's fibre, not starch or sugar; Wikipedia actually seems to indicate it's used to
stop bacterial growth in cultures (because unlike normal liquid growth medium, it's too thick for them to travel though). I certainly haven't seen mould or anything. I may have swallowed a little two-week old, particularly lumpy, chunky gunge a few weeks ago, and there were no noticeable ill effects (beyond tasting bloody 'orrible and making me want to gag)!
But ... no, don't try and keep it once mixed up for extended periods, because you want a
pleasant experience when sploshing!
In summary—go for it. It's awesome.

But you probably don't want to scoop it up and re-use it a week later.
A few people sell it quite cheaply on eBay now, including Dave from gungetankbabes, I believe. It's cheap enough you shouldn't need to re-use it. I mean, you don't reuse your custard and beans, do you...?
...y-you don't, do you? *winces*