How to Analyze and Visualize Marks & Spencer's Nightwear Products via Web Scraping?

Introduction

Marks & Spencer is an iconic British brand celebrated for its extensive product spectrum, encompassing top-tier clothing, gourmet food, home essentials, premium beauty items, and a robust ethos centered on sustainability and societal impact. This eclectic blend has solidified M&S's stature as a favored destination for shoppers in the United Kingdom.

In this article, leveraging web scraping services, we delve into the nuances of Marks & Spencer's nightwear collection, harnessing data analytics services to shed light on:

• The product price distribution across various brands.
• A comprehensive brand-wise product count analysis.
• The most esteemed brands are based on customer reviews and trust.
• The average product price of leading brands juxtaposed with their respective review counts.
• Insights into prevalent discounts and sales trends across brands.

Through this Market Research-driven exploration, readers will gain a holistic view of the nightwear segment at M&S, offering invaluable insights for consumers and industry enthusiasts.

Web Scraping: The Concept

Web scraping is the cutting-edge technique employed to extract pertinent insights from websites. Although manual data collection remains an option, it often proves tedious, time-intensive, and susceptible to inaccuracies. In contrast, data scraping services offer a streamlined, swift, and precise approach to automate data extraction.

A standout feature of data scraping lies in its adeptness at capturing either non-tabular data or lacking a defined structure from websites. Such data is meticulously transformed through this method into organized, user-friendly formats, be it spreadsheet or .csv files. Such structured data becomes invaluable, especially when harnessed by data analytics services for comprehensive analysis or integrated into market research endeavors.

Beyond its foundational data collection role, web scraping emerges as a pivotal instrument for data archival and real-time monitoring of online data repositories. By harnessing its capabilities, businesses and analysts can effortlessly monitor product price fluctuations, track dynamic content changes, and make sure they consistently access the most current information for informed decision-making.

Data Collecting Process

We used our in-house web scraping platform for collecting publicly accessible data from Marks & Spencer as an initial exploration of nightwear products. Python, along with a BeautifulSoup library, was used for the seamless extraction of this data using web scraping techniques.

After extracting the data, it was efficiently stored in a CSV format making use of a Pandas library. This CSV dataset was integrated in the SQLite database for bolstering data management capabilities.

The data types of column field were adjusted meticulously to align with their respective attributes and values to optimize the database structure. This streamlined subsequent querying and analysis processes.

To further enhance our analytical capabilities, we seamlessly integrated the SQLite database with Metabase, a renowned business intelligence & analytics platform. This allowed us to craft insightful analytics charts effortlessly, selectively adding pertinent fields and judiciously applying data filters to distill actionable insights.

Attributes Extracted from Marks & Spencer Lingerie Products

• Brand Name: Identifies the product's manufacturer.
• Title: Specifies the product's category or type.
• Product URL: Direct link to the product's webpage.
• Average Rating: Mean value of all product ratings.
• Product Code: A unique identifier assigned to all products.
• Reviews: Total count of customer reviews for a product.
• Selling Price: Current cost at which the product is available.
• Original Price: Manufacturer's suggested retail price (MSRP) for the product.
• Discount: Calculated reduction between original and selling prices.
• Sales Status: Indicates if the product is currently on sale.
• Configuration: Lists the materials or components used in the product's manufacture.
• The Python script employed leverages several essential libraries, each pivotal in the intricate process of data scraping and data management:
• Requests: A cornerstone to make HTTP requests, this library facilitates interactions with webpages, which is pivotal for web scraping services.
• Beautiful Soup: This tool specializes in analyzing XML and HTML structures. Combined with Requests, it provides a robust solution for data extraction, crucial for data analytics services.
• Pandas: Renowned for its DataFrames, Pandas offers a versatile platform for data manipulation. It's instrumental in organizing the scraped product data, which is essential for market research and product price analysis.
• Regular Expressions ('re'): This built-in library offers pattern-matching capabilities, ensuring precise data extraction, which is especially beneficial for extracting product prices or reviews.
• OS Library: The integral component for file operations, enabling efficient data storage and management, vital for organizing the vast amounts of data collected during web scraping.
• RequestException: This specialized class aids in handling HTTP request errors, ensuring the script remains robust and reliable during data collection, a key feature for web scraping services.

Integrating these libraries ensures a seamless process, from web scraping to data analytics services, facilitating comprehensive market research and product price evaluation.

Establishing Data Storage Structure

To systematically organize and manage product details, a series of lists have been set up. These lists capture crucial information for each product, encompassing:

• URL
• Brand Name
• Product Name
• Product Code
• Average Rating
• Customer Reviews Count
• Selling Price
• Manufacturer's Suggested Retail Price
• Discount Amount
• Color Variants
• Sales Availability
• Product Styles
• Materials Used (Composition)

This structured approach ensures that all pertinent data related to the products is efficiently cataloged and readily accessible for subsequent analysis or processing.

Script Functions Overview

Within the script, a suite of functions has been meticulously crafted to address distinct tasks, ranging from fetching the HTML content of a given URL to extracting intricate product information like the brand and product name.

Core Functionality - Main Function:

Serving as the pivotal hub of the script, the primary function orchestrates the intricate ballet of data scraping. It starts by establishing foundational parameters: specifying base URLs and configuring the necessary headers tailored to make HTTP requests for an M&S website.

The primary phase involves invoking the extract_href_values function to procure the HTML representation of the main product page. When this foundational dataset is in place, the primary function dynamically determines the total pagination count, crafting the URLs for every page. Armed with this curated list of URLs, the function employs the function extract_product_url for pinpointing and extracting the unique URLs corresponding to individual products.

The culmination of this orchestrated process is the pivotal fetch_product_details function. This function delves deep, meticulously scraping and collating nuanced product attributes.

The primary function is the linchpin, seamlessly guiding the script through its various stages, culminating in a comprehensive dataset extracted from M&S digital storefront.

Product Details Extraction

Within fetch_product_details functions, a script systematically processes every product URL. It initiates an HTTP request for accessing a product page, harvesting crucial attributes like brand, product title, unique code, user ratings, customer reviews, pricing details, color variants, sales availability, and product composition. Each attribute is methodically appended to the designated lists for organized storage.

Data Compilation & Export

After the exhaustive scraping of product data, the script leverages the prowess of a Pandas library. It molds the accumulated information into a structured DataFrame, ensuring clarity and coherence. Before committing this structured data to a file, a preliminary check is conducted to inspect the existence of the file called 'mas.csv.' Should such a file be detected, it's seamlessly updated with the fresh dataset. Conversely, the script crafts a pristine CSV file without such a file. The resultant DataFrame, enriched with product nuances, is then archived into the 'mas.csv', cementing its accessibility for subsequent analytical endeavors, reporting tasks, or diverse data-centric operations.

Script Invocation:

A conditional directive within the script discerns its execution mode. If the script is invoked directly (and not imported like a module), it autonomously triggers the primary function, setting the web scraping mechanism into motion.

In a Nutshell:

A Python script is meticulously engineered to delve into the depths of M&S's nightwear and lingerie offerings. It adeptly extracts many product attributes, subsequently encapsulating them in a structured format within the CSV file, priming the data for in-depth analysis, archival, or further computational tasks.

Product Information Extraction Mechanism

The script intricately lays out a suite of functions tailored to harness a spectrum of data points from product-centric web pages. These cohesive functions synergize to unearth pivotal details, including brand identity, product nomenclature, pricing specifics, user feedback, color palettes, and various product attributes. Here's a detailed breakdown of each function's modus operandi:

Web Request & Parsing: The vanguard is the function of the extract_href_values(headers, URL, max_retries=3). It adeptly dispatches the GET request for the chosen URL, fortified with specified headers. Moreover, with built-in resilience, it retries thrice upon encountering networks or requests anomalies. The function culminates by furnishing the HTML content, meticulously parsed via Beautiful Soup.

Pagination Insights: With extract_number_of_pages(soup), the script deciphers the HTML's anatomy, zeroing in on an element encapsulating pagination data. The outcome? A crisp integer denoting the maximum pagination limit.

Product URL Compilation: The extract_product_url(pageUrls, baseUrl, headers) function amalgamates base and page URLs. Its iterative prowess scours each page, harvesting product URLs en route, ultimately presenting a curated list.

Brand & Product Names: While extract_company is trying to spotlight brand names using discerning classes, extract_product_name is hot on the trail of the product's nomenclature. Both functions yield their respective findings or a "Not available" tag.

Product Code & Ratings: A product code, a unique identifier, is the purview of extract_product_code. Concurrently, extract_avg_rating zeroes in on the average user rating, with both functions either delivering the data or signaling its absence.

User Feedback Metrics: While extract_reviews quantifies the feedback tally, extract_selling_price and extract_original_price navigate pricing intricacies, ensuring data accuracy and consistency. Meanwhile, extract_saved_price decodes discount nuances.

Product Attributes: From extract_color spotlighting product hues to extract_sales_status deciphering sales dynamics, the script leaves no stone unturned. An extract_composition function delves into material specifics, ensuring a holistic product profile.

This meticulously crafted ensemble of functions epitomizes the pinnacle of data scraping services. They collaboratively empower the script, ensuring a comprehensive harvest of product insights—a boon for data analytics services, market research endeavors, and granular product price analysis.

Store Product Details

The Python script captures details of nightwear and lingerie items from M&S website, systematically extracting specific attributes for each product. Subsequently, this data is saved into the CSV file, setting the stage for in-depth analysis or archival purposes.

Gleaning Insights from the Acquired Data:

1. Brand Analysis Based on Frequency.

The following presents a brand-wise analysis, gauging each brand's presence based on product count:

• "Body" stands out with 42 products, emphasizing bodywear. Meanwhile, "Seasalt Cornwall" introduces a specific range with 10 products.
• "Cyberjammies" trails closely behind at 120 products, emphasizing its expertise in sleepwear.
• “M&S X GHOST" and "DKNY" are tied having 21 products each, symbolizing a blend of global prominence and distinct collaborative designs.
• "FatFace" and "Rosie" are neck and neck with 8 products each, suggesting varied casual and sleepwear options.
• "Hotmilk," "Elomi," and "Triumph" are represented with 2 products each, hinting at specialized segments. Lastly, "Autograph," "Kate Spade," and "Percy Pig™" register the least counts, pointing toward unique or limited-edition collections.
• "Nobody's Child" offers a distinctive range having 7 products.
• Brands such as "Spencer Bear™", "Boutique," and "Fantasie," are on the niche side with fewer counts, possibly signifying exclusivity.
• Lingerie-focused "Wacoal" lists 5 products.
• The "M&S Collection" is leading the charts with a robust 133 products, underlining Marks & Spencer's broad offerings.

This analysis provides a comprehensive view of brand positioning, product variety, and market trends within the apparel sector.

Brands with the Highest Trust and Reviews:

Top Brands Based on Customer Reviews:

M&S Collection: Leading the charts, M&S Collection boasts the highest review count, reflecting its widespread popularity and customer trust. This brand resonates with a large segment of consumers.

Body: Following closely, Body has amassed many reviews, emphasizing its relevance and appeal in the bodywear market. This brand's products capture substantial consumer attention.

Cyberjammies: Specializing in sleepwear, Cyberjammies has garnered many reviews, showcasing its strong foothold in the niche pajama segment.

Spencer Bear™: With notable reviews, Spencer Bear™ demonstrates a unique charm or perhaps a dedicated fan base, given its likely association with a specific product range.

Rosie: Earning a moderate review count, Rosie captures a decent level of consumer interest, possibly within the sleepwear or lingerie domain.

Percy Pig™: Despite not being a traditional clothing brand, Percy Pig™ had secured a reasonable total reviews, hinting at its distinct market presence or product collaboration.

M&S X GHOST: This collaborative venture has attracted many reviews, underscoring the intrigue surrounding its combined product offerings.

Wacoal: With a moderate review count, Wacoal, known for its undergarments and lingerie, maintains a consistent but perhaps more niche customer feedback level.

Boutique: The boutique, likely with its specialized product range, has a reasonable review count, indicating steady engagement having its offerings.

This review-based ranking provides insights into brand performance and consumer sentiment. While M&S Collection leads, each brand's review count underscores its position and resonance in the market.

Average Ratings of Leading Brands Relative to Review Counts

Body consistently receives top ratings, highlighting excellent customer satisfaction despite a modest product lineup.

M&S Collection boasts the most extensive product assortment and generally favorable ratings, indicating a broad selection that aligns with customer preferences.

M&S X GHOST continues to impress with its high ratings, reflecting a positive reception for its joint offerings.

While offering fewer products, Seasalt Cornwall maintains a commendable average rating.

Cyberjammies offers many products but could benefit from enhancing customer satisfaction.

DKNY, with its decent product range, grapples with the challenge of the lowest average rating, signaling a need to enhance the customer experience.

This assessment offers valuable insights for businesses to gauge their performance against their product offerings and customer feedback, paving the way for informed decisions on enhancement or diversification.

Average Savings on Brands Currently Discounted

Nobody's Child leads in offering the highest average discount prices, making their sale items especially appealing for savvy shoppers. We found that Kate Spade lists a standout discounted product using advanced web scraping services, though data analytics services reveal its limited availability. Cyberjammies showcases diverse products with competitive average discount rates, meeting varied market research insights on consumer demand. Meanwhile, with its narrow product selection and minimal average discount, Boutique might benefit from enhanced product pricing strategies. This comprehensive analysis, driven by meticulous data analytics services and market research, sheds light on how companies strategically price and discount their offerings to cater to diverse customer preferences.

Conclusion

Our comprehensive data analysis, powered by cutting-edge web scraping and data analytics services, unveils crucial insights into the competitive landscape of product pricing, offerings, and customer engagement across various brands.

"Nobody's Child" emerges as a frontrunner, presenting the most attractive product price discounts, resonating well with cost-conscious consumers. Despite a narrower product scope, "Kate Spade" impressively offers significant discounts, a revelation brought to light through meticulous market research. "Cyberjammies," backed by extensive market research insights, showcases a broad product portfolio with moderate price reductions, appealing to a broad audience. Conversely, "Boutique" lags with minimal discounts on its limited product line.

Among the brands analyzed, "M&S Collection" stands tall, blending a diverse product range with stellar customer satisfaction ratings. In contrast, despite its expansive product array, "Cyberjammies" grapples with suboptimal customer satisfaction metrics. The brand "Body" carves a niche in bodywear, boasting top-tier customer approval ratings. Meanwhile, niche players like "M&S X GHOST," "Seasalt Cornwall," and "Nobody's Child" bring their distinct product offerings to the fore.

Delving into customer engagement metrics, "M&S Collection" shines brightly, buoyed by many reviews. Brands like "Cyberjammies" and "Rosie" also exhibit commendable customer interaction. However, "Boutique" and "Wacoal" appear less prominent, possibly attributed to their specialized market segments.

For businesses eager to harness such invaluable market insights to refine their strategies, Actowiz Solutions offers unparalleled expertise. Whether it's web scraping, data analytics, or market research, we empower brands to bolster their competitive stance. Elevate your market understanding; partner with Actowiz Solutions today. You can also contact us for all your requirements related to You can also reach us for all your mobile app scraping, instant data scraper and web scraping service requirements.