UbuWeb was gone but now it's back. This vast collection of strange spoken word, audio art, sound poetry, and written nonsense is completely invaluable.
The Live Music Archive is the premier place to share legally recorded live music. It also has lots of historical and spoken-word recordings. For example, there are conference proceedings, 300 recordings from the Jack Kerouac School of Disembodied Poetics, and historical artifacts from as long ago as 1890.
Top 10 Longest Waits For A Decent Album is one of the funniest musical articles you will ever read. There are others nearby as well at this Wire-related (always a good sign) site.
Alarm Will Sound have done a cover version of Aphex Twin songs by playing all of the parts on classical instruments. The loons.
disquiet contains "reflections on ambient/electronic music, and interviews with the people who make it" and lots of links to free downloads as well.
Mashups are strange not-entirely-illegal collisions of popular songs. Examples from dINbOT and bittersoundfoundation have made it into my mixes.
You may want to hear an interview with Brian Eno about his mostly disappointing recent album Another Day on Earth. Hmmm... why is Brian Eno now more interesting than his music? This is a podcast1 by the way.
Joey Honey is an odd and perhaps idiotic person who has some strange electro tunes scattered around his site.
Ampersand Etc. contains illegible reviews of music that is mostly kinda industrial.
Newcomers will find other suggestions in the article Field Recording Resources.
1 What is a podcast? A podcast is what idiots call an audio file that likely contains a mediocre collection of music mixed with a college-radio-bad announcer -- some muttering nutter recorded with a came-with-the-computer microphone plugged directly into their SoundBlitzer sound card. It's called "pod" because it's made for the sort of Invasion of the Body Snatcher pod people who buy iPods and then set up class action lawsuits because the surface scratches. It's called "cast" because it's like a radio broadcast, except that it's not radio and not broadcast, so that it kind of, um, well, isn't like a radio broadcast at all. OK, rant over. [Geez, I just realised I never rant on this site. What's a blog for anyway?]