Alex Mathew

Product Engineer. Code writer. Python enthusiast. Sports fan.

I Have to Talk About Scrapple

scrapple, web scraping

*One and a half years after my last blog post*

I maintain a project on Github, and I can’t shut up about it. I won’t deny that I talk about it a lot, and that’s because I’m proud of it. But I still don’t have a proper way of explaining it to people. I think I have to start writing about what it can do.

I started working on Scrapple in late 2014 (which was about the time I wrote my last blog post before this). It was an idea that was running on my mind for a while - an itch that I wanted to scratch. The snowball effect struck, and in a few months, Scrapple was sort of a big deal on Github (for a day) and I felt this unusual sense of responsibility that I had to explain to people what exactly it was that I was trying to achieve. For example, I gave a talk at a Python users meetup about Scrapple. I have met people who could relate to the problem and have given me some great advice and suggestions. To the other people who could not relate to it, I have tried explaining the idea of Scrapple.

And I have failed. Quite miserably.

I have tried to maintain an extensive documentation of the entire code base. But in terms of people trying to use Scrapple, this does not particularly help. These people want to know how Scrapple can be useful to them, how to get started with using it, and the various options they have at their disposal. The couple of examples that I have added to the Scrapple documentation were definitely not enough for all this. I had to do more.

Add this need with my decision to try to start writing and the fact that I have no real way to maintain my blog so far, and I think I have a solution. Write about web scraping in general, and how Scrapple fits in.

I might manage to get my blog alive, get a clearer picture of what I am trying to achieve with Scrapple, and have an overall better understanding of maintaining a project and getting people involved in it. And have fun doing it.

I don’t know how often I’ll manage to write, or what exactly I will be writing. I’ll let this take shape in time.

Hopefully, it works out well.