How to Get Data from a Fashion Website using Python & Beautiful Soup?

Web scraping basics

Initially, let's understand some concepts about data scraping. Web scraping is a procedure used to scrape a massive amount of data from websites to create data sets.

We perform this by using a website's source codes and scraping the required data. The complicated part here is understanding how a website's source codes get structured.

Website and HTML

Websites are created using HTML, a standard markup language. HTML is a formless format that relates data with particular elements.

Every website has a precise structure. Think of it as boxes or containers. Each container holds a website section having images, videos, text, or other containers.

The initial thing you have to do is understand which container has the information you need to fetch. For that, you must locate an HTML tag with the data you want.

Web designers are using HTML tags like " h1 , span , class , and p " for classifying content and style. You will get a listing of HTML tags here.

1. Getting a website's source code

You can review a website by right-clicking on a section and choosing an option called "Inspect." Your browser would open a tiny window with a site's HTML code, highlighting the name section where targeted content is saved.

Here, we want product name and pricing data. A product name gets stored on the tag with a class "product-detail-card-info__name." You could save this data by right-clicking the code section you need to scrape and choosing Copy-> Copy outside HTML.

2. Use beautiful Soup for fetching data from websites

Now as we understand where data is saved on the website, the following step is scraping content and keeping that in the excellent data frame.

Initially, we load libraries which we will use here:

• requests: Permits us to dispatch requests to a website URL.
• pandas: Utilized to analyze and make well-structured data.
• bs4: A library that permits us to extract data from sites.
• Export data into the CSV file

Request data from websites

1. Getting a website's source code

We initially set a website URL we need to extract as a variable.

After that, we will send the request to a website for fetching data.

And utilize Beautiful Soup for scraping a page's HTML code.

After that, we scrape labels where the content we wish is. Here, product names are saved on the h3 tags, and pricing data is stored in the span tags underneath a class name.

The complete code to scrape a website is given below:

3. Clean the results

The following step is storing data in the Pandas data frame; therefore, we organize the scraped data.

Any scraped data from the website using BeautifulSoup is saved as a BeautifulSoup element, similar to < class' bs4.element. ResultSet'>. We have to change that to data types that could be held on the pandas Dataframe, identical to a dictionary or list.

We also have to ensure that data gets clean before passing that to Pandas' data frame.

Scraping text

We can scrape text from BeautifulSoup elements and save that as a listing using the following code:

While exploring the results of the given lists, we could find that a few list elements aren't a part of the data we wish to scrape. Passing data to the text format doesn't work as needed. Therefore, we make a listing crunching only the information we want.

As we have to clean data for different names, we make a new listing with a string, including HTML tags. We create a new listing and get only the elements that we need. Then, we eliminate an HTML tag from outstanding features on a list. Here, we will utilize a for loop, which excludes elements having HTML tags containing the word "class."

4. Use of Pandas to well-structure data

Once the data is clean, we pass each list like a column of the Pandas' data frame.

The final step is saving a data frame in the CSV format.

And that's it! We're done! If you have enjoyed this blog and want to know more, contact Actowiz Solutions now! You can also reach us for all your mobile app scraping and web scraping services requirements.