Adding New Blogs
Some of you have noticed an increasing number of blogs added to the index in the past week (click the image for a larger view).
While the impact isn't huge, yet, it has the potential to grow very quickly. Last Spring, our former Admin (and still friend) AdamF developed a solution called CleanFeed. The basic concept is to take in the content of the ping cloud and sort out the wheat from the chaff. SubWolf and I have been working over the last couple of weeks to tidy up and get CleanFeed online, permanently. It's moving along pretty nicely, but there are still some limitations on what the system can do.
The numbers are pretty staggering. In the course of a few minutes, we grab 30-50 thousand pings from the cloud. About 9% of those appear to be "good" blogs. Our blacklists screen out 46%, and unsafe file extensions or banned top level domains filter out 31%. We also see 6% with no dates or not enough posts, 6% time out, and 3% return an error when CleanFeed tries to read them.
For now, you should see an average of 10 entries per minute added to the index. You'll have lots of opportunities for chips, karma and sigma from voting these new blogs and killing off our mistakes. If you see any patterns emerging, please let us know so we can tweak the filters.
--
Ken