Creating my first CLI HTML scraping Ruby Gem

Posted by craig wadding on September 26, 2019

Before I describe the first CLI project I have done at Flatiron School, let me give some good advice before you start one yourself.

  1. When the instructors recommend you search for a good site and make sure it is something you can scrape for good data, take it seriously. Don’t be afraid to check out a few dozen websites before you decide. There are several videos available on what to look for and why, like this one https://instruction.learn.co/student/video_lectures#/329 .

  2. Work on getting requirements to passing the assessment done first, once you do , then go back if you want to make the project more interactive, supply more options or scrape more levels. Don’t bite off more than you can chew, depending on which track you are enrolled in , your time maybe as short as 1 week to complete your project.

  3. DON’T BE AFRAID TO ASK FOR HELP! You may get stuck, heck you probably will get stuck…….don’t beat yourself up for 3 days, ask a friend, ask a cohort member, as an instructor…..you will find there are usually many willing to help.

Had I followed my own advice above, I would not have found myself stuck on the Nokogiri section of my scraper class picking the correct CSS selector to get what I needed. The site I picked was poorly done and the creator’s name must have been David Div, from Divburgh in Div County, because he sure loved using Div’s.

In any case, I was all in, the clock was ticking…..I needed to get finished, most of the ideas of what I wanted my project to be were “cut on the editing room floor”. I mean, I finished it , and it works….. let’s hope I pass the technical assessment.

What did I learn? Well for one thing, when you have mystery ghost instances in your array, “.drop” in Ruby really comes in handy. “Ghosts?” ….long story, but the .drop busted them ghosts.

So yes, my CLI project for Flatiron is simple and basic but does it pass all of the requirements? lets see, shall we?

  • Provide a CLI ? check
  • Provide access to data froma website ? check
  • Provided data must go at least 1 “level” deep? check
  • Should not be similar to other CLI projects from curriculum? check
  • Use good OO design patterns. Avoid scraping data more than once per page? ….. cough ….check *“good OO design patterns” , let’s hope so)

So how does it work? you run the bin file, you see a list of available doggos to adopt, you select a doggo and you get said doggo’s information. Could it have been better? Oh yes, and I wish it was. I think I may visit my gem again soon and make it all pretty, maybe try to put some sparkle to it…but for now I’m glad I’m learning and I’m finished with my first project. I mean, isn’t that why we are here? I’ll be 47 years old this weekend. I’ve learned you take the victories, even the small ones.

On to the next one, Sinatra? Active Record? ORM …..ORM who? SQL? bring it on!

Craig Wadding FT081219