Category-wise packs with monthly refresh; export as CSV, ISON, or Parquet.
Choose your region, and we’ll deliver clean, accurate store location datasets.
Launch instantly with ready-made scrapers tailored for popular platforms. Extract clean, structured data without building from scratch.
Access real-time, structured data through scalable REST APIs. Integrate seamlessly into your workflows for faster insights and automation.
Download sample datasets with product titles, price, stock, and reviews data. Explore Q4-ready insights to test, analyze, and power smarter business strategies.
Playbook to win the digital shelf. Learn how brands & retailers can track prices, monitor stock, boost visibility, and drive conversions with actionable data insights.
We deliver innovative solutions, empowering businesses to grow, adapt, and succeed globally.
Collaborating with industry leaders to provide reliable, scalable, and cutting-edge solutions.
Find clear, concise answers to all your questions about our services, solutions, and business support.
Our talented, dedicated team members bring expertise and innovation to deliver quality work.
Creating working prototypes to validate ideas and accelerate overall business innovation quickly.
Connect to explore services, request demos, or discuss opportunities for business growth.
GeoIp2\Model\City Object ( [raw:protected] => Array ( [city] => Array ( [geoname_id] => 4509177 [names] => Array ( [de] => Columbus [en] => Columbus [es] => Columbus [fr] => Columbus [ja] => コロンバス [pt-BR] => Columbus [ru] => Колумбус [zh-CN] => 哥伦布 ) ) [continent] => Array ( [code] => NA [geoname_id] => 6255149 [names] => Array ( [de] => Nordamerika [en] => North America [es] => Norteamérica [fr] => Amérique du Nord [ja] => 北アメリカ [pt-BR] => América do Norte [ru] => Северная Америка [zh-CN] => 北美洲 ) ) [country] => Array ( [geoname_id] => 6252001 [iso_code] => US [names] => Array ( [de] => USA [en] => United States [es] => Estados Unidos [fr] => États Unis [ja] => アメリカ [pt-BR] => EUA [ru] => США [zh-CN] => 美国 ) ) [location] => Array ( [accuracy_radius] => 20 [latitude] => 39.9625 [longitude] => -83.0061 [metro_code] => 535 [time_zone] => America/New_York ) [postal] => Array ( [code] => 43215 ) [registered_country] => Array ( [geoname_id] => 6252001 [iso_code] => US [names] => Array ( [de] => USA [en] => United States [es] => Estados Unidos [fr] => États Unis [ja] => アメリカ [pt-BR] => EUA [ru] => США [zh-CN] => 美国 ) ) [subdivisions] => Array ( [0] => Array ( [geoname_id] => 5165418 [iso_code] => OH [names] => Array ( [de] => Ohio [en] => Ohio [es] => Ohio [fr] => Ohio [ja] => オハイオ州 [pt-BR] => Ohio [ru] => Огайо [zh-CN] => 俄亥俄州 ) ) ) [traits] => Array ( [ip_address] => 216.73.216.157 [prefix_len] => 22 ) ) [continent:protected] => GeoIp2\Record\Continent Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [code] => NA [geoname_id] => 6255149 [names] => Array ( [de] => Nordamerika [en] => North America [es] => Norteamérica [fr] => Amérique du Nord [ja] => 北アメリカ [pt-BR] => América do Norte [ru] => Северная Америка [zh-CN] => 北美洲 ) ) [locales:GeoIp2\Record\AbstractPlaceRecord:private] => Array ( [0] => en ) [validAttributes:protected] => Array ( [0] => code [1] => geonameId [2] => names ) ) [country:protected] => GeoIp2\Record\Country Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [geoname_id] => 6252001 [iso_code] => US [names] => Array ( [de] => USA [en] => United States [es] => Estados Unidos [fr] => États Unis [ja] => アメリカ [pt-BR] => EUA [ru] => США [zh-CN] => 美国 ) ) [locales:GeoIp2\Record\AbstractPlaceRecord:private] => Array ( [0] => en ) [validAttributes:protected] => Array ( [0] => confidence [1] => geonameId [2] => isInEuropeanUnion [3] => isoCode [4] => names ) ) [locales:protected] => Array ( [0] => en ) [maxmind:protected] => GeoIp2\Record\MaxMind Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( ) [validAttributes:protected] => Array ( [0] => queriesRemaining ) ) [registeredCountry:protected] => GeoIp2\Record\Country Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [geoname_id] => 6252001 [iso_code] => US [names] => Array ( [de] => USA [en] => United States [es] => Estados Unidos [fr] => États Unis [ja] => アメリカ [pt-BR] => EUA [ru] => США [zh-CN] => 美国 ) ) [locales:GeoIp2\Record\AbstractPlaceRecord:private] => Array ( [0] => en ) [validAttributes:protected] => Array ( [0] => confidence [1] => geonameId [2] => isInEuropeanUnion [3] => isoCode [4] => names ) ) [representedCountry:protected] => GeoIp2\Record\RepresentedCountry Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( ) [locales:GeoIp2\Record\AbstractPlaceRecord:private] => Array ( [0] => en ) [validAttributes:protected] => Array ( [0] => confidence [1] => geonameId [2] => isInEuropeanUnion [3] => isoCode [4] => names [5] => type ) ) [traits:protected] => GeoIp2\Record\Traits Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [ip_address] => 216.73.216.157 [prefix_len] => 22 [network] => 216.73.216.0/22 ) [validAttributes:protected] => Array ( [0] => autonomousSystemNumber [1] => autonomousSystemOrganization [2] => connectionType [3] => domain [4] => ipAddress [5] => isAnonymous [6] => isAnonymousProxy [7] => isAnonymousVpn [8] => isHostingProvider [9] => isLegitimateProxy [10] => isp [11] => isPublicProxy [12] => isResidentialProxy [13] => isSatelliteProvider [14] => isTorExitNode [15] => mobileCountryCode [16] => mobileNetworkCode [17] => network [18] => organization [19] => staticIpScore [20] => userCount [21] => userType ) ) [city:protected] => GeoIp2\Record\City Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [geoname_id] => 4509177 [names] => Array ( [de] => Columbus [en] => Columbus [es] => Columbus [fr] => Columbus [ja] => コロンバス [pt-BR] => Columbus [ru] => Колумбус [zh-CN] => 哥伦布 ) ) [locales:GeoIp2\Record\AbstractPlaceRecord:private] => Array ( [0] => en ) [validAttributes:protected] => Array ( [0] => confidence [1] => geonameId [2] => names ) ) [location:protected] => GeoIp2\Record\Location Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [accuracy_radius] => 20 [latitude] => 39.9625 [longitude] => -83.0061 [metro_code] => 535 [time_zone] => America/New_York ) [validAttributes:protected] => Array ( [0] => averageIncome [1] => accuracyRadius [2] => latitude [3] => longitude [4] => metroCode [5] => populationDensity [6] => postalCode [7] => postalConfidence [8] => timeZone ) ) [postal:protected] => GeoIp2\Record\Postal Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [code] => 43215 ) [validAttributes:protected] => Array ( [0] => code [1] => confidence ) ) [subdivisions:protected] => Array ( [0] => GeoIp2\Record\Subdivision Object ( [record:GeoIp2\Record\AbstractRecord:private] => Array ( [geoname_id] => 5165418 [iso_code] => OH [names] => Array ( [de] => Ohio [en] => Ohio [es] => Ohio [fr] => Ohio [ja] => オハイオ州 [pt-BR] => Ohio [ru] => Огайо [zh-CN] => 俄亥俄州 ) ) [locales:GeoIp2\Record\AbstractPlaceRecord:private] => Array ( [0] => en ) [validAttributes:protected] => Array ( [0] => confidence [1] => geonameId [2] => isoCode [3] => names ) ) ) )
country : United States
city : Columbus
US
Array ( [as_domain] => amazon.com [as_name] => Amazon.com, Inc. [asn] => AS16509 [continent] => North America [continent_code] => NA [country] => United States [country_code] => US )
In Part 1 of the two-part series on data scraping e-commerce websites for price comparison, we used the Selenium-Python package to automate the procedure of extracting product prices and names from the Lazada website.
In Part 2 here, we will continue scraping on the Shopee website. Here we will concentrate on particular challenges with extracting the Shopee website rather than repeating the steps in Part I. We will also introduce a substitute to Selenium that worked better!
So, let’s begin!
wasn’t easy while using a Selenium tool, and we have highlighted four extra complexities a Shopee website had and a Lazada website hadn’t:
Popup Alerts (Extra Complexity = Low)The initial issue we meet is popup alerts, which come when you search:
We can automate clicking away from popup boxes using Selenium with the given script:
We also get that at times in Shopee search outcomes, one item might have two different pricing figures with a similar class name. Different prices imitate a pricing range where an item has a volume discount:
Using Selenium, we can stipulate the particular figure we need by using an XPath selector to choose the second span component that reflects the initial figure:
The Shopee site is a dynamic site, where page elements look dynamically only while scrolling down a page. It isn’t unusual because it helps a page in loading quicker without immediately loading all elements (Facebook works in the similar way).
However, this needs to automate scrolling to bottom of a page like you would do physically, with shorter waiting time for all page fundamentals to appear.
Also, Selenium allows automation to do browser scrolling, however the script for the particular automation could be lengthy because you might need to imitate the manual procedure of scrolling a bit more, and wait a few seconds for page elements to come, rinse and repeat till you reach end of a page.
We could write the script like this:
Here we can see that the code has become much more composite, and the automation procedure has also become slower with extra pause times.
As observed earlier, the product names can’t be selected although they could be recognized with either XPath or class selectors and could be seen with a Chrome inspect tool. Due to that, running find_element doesn’t reoccurrence the anticipated item names, only empty strings.
We’ll have to write a few Javascript codes to deploy a CSS property, the language we are extremely unfamiliar with.
Fortunately, we found an easy way of scraping Shopee data: using Shopee’s API to ask for search results.
We were extremely lucky to find that on the web. Not all the websites will get or will share the API with you. Because Shopee helps you use the API to extract product information directly, it becomes much easier to utilize that rather than automating the extraction procedure with Selenium using the given code:
Now, we will make a pandas dataframe for organizing all the data:
Printing output data of a dataframe offers the given results:
With a Lazada dataset, we would also require to conduct cleaning with the dataset. The key things we have to do include:
Now, it’s time to combine Shopee and Lazada datasets! We do that by utilizing a pandas concatenation technique:
Now we need to compare between these two platforms. We could print a dataframe statistical structures using a describe method:
We would plot data using similar box plot created in the Part 1:
And that’s it! Depending on one item comparison, it does look that Shopee is the cheaper platform (having extra items).
Some notes before we finish off:
a) It’s useful to organize a price comparison between various time periods to analyze a pricing trend of any particular item. To do that, we could add a datetime column as well as save this to the csv file.
b) Though you can extract other items just by changing a keyword_search variable, you might have to clean a dataset otherwise from the given example.
c) This example is the small dataset, and so the cleaning and scraping exercise was much quicker.
That’s it for now!
For more information about scraping e-commerce website data to compare prices using Python, contact Actowiz Solutions now!
You can also reach us for all your mobile app scraping and web scraping services requirements.
✨ "1000+ Projects Delivered Globally"
⭐ "Rated 4.9/5 on Google & G2"
🔒 "Your data is secure with us. NDA available."
💬 "Average Response Time: Under 12 hours"
Look Back Analyze historical data to discover patterns, anomalies, and shifts in customer behavior.
Find Insights Use AI to connect data points and uncover market changes. Meanwhile.
Move Forward Predict demand, price shifts, and future opportunities across geographies.
Industry:
Coffee / Beverage / D2C
Result
2x Faster
Smarter product targeting
“Actowiz Solutions has been instrumental in optimizing our data scraping processes. Their services have provided us with valuable insights into our customer preferences, helping us stay ahead of the competition.”
Operations Manager, Beanly Coffee
✓ Competitive insights from multiple platforms
Real Estate
Real-time RERA insights for 20+ states
“Actowiz Solutions provided exceptional RERA Website Data Scraping Solution Service across PAN India, ensuring we received accurate and up-to-date real estate data for our analysis.”
Data Analyst, Aditya Birla Group
✓ Boosted data acquisition speed by 3×
Organic Grocery / FMCG
Improved
competitive benchmarking
“With Actowiz Solutions' data scraping, we’ve gained a clear edge in tracking product availability and pricing across various platforms. Their service has been a key to improving our market intelligence.”
Product Manager, 24Mantra Organic
✓ Real-time SKU-level tracking
Quick Commerce
Inventory Decisions
“Actowiz Solutions has greatly helped us monitor product availability from top three Quick Commerce brands. Their real-time data and accurate insights have streamlined our inventory management and decision-making process. Highly recommended!”
Aarav Shah, Senior Data Analyst, Mensa Brands
✓ 28% product availability accuracy
✓ Reduced OOS by 34% in 3 weeks
3x Faster
improvement in operational efficiency
“Actowiz Solutions' data scraping services have helped streamline our processes and improve our operational efficiency. Their expertise has provided us with actionable data to enhance our market positioning.”
Business Development Lead,Organic Tattva
✓ Weekly competitor pricing feeds
Beverage / D2C
Faster
Trend Detection
“The data scraping services offered by Actowiz Solutions have been crucial in refining our strategies. They have significantly improved our ability to analyze and respond to market trends quickly.”
Marketing Director, Sleepyowl Coffee
Boosted marketing responsiveness
Enhanced
stock tracking across SKUs
“Actowiz Solutions provided accurate Product Availability and Ranking Data Collection from 3 Quick Commerce Applications, improving our product visibility and stock management.”
Growth Analyst, TheBakersDozen.in
✓ Improved rank visibility of top products
Real results from real businesses using Actowiz Solutions
In Stock₹524
Price Drop + 12 minin 6 hrs across Lel.6
Price Drop −12 thr
Improved inventoryvisibility & planning
Actowiz's real-time scraping dashboard helps you monitor stock levels, delivery times, and price drops across Blinkit, Amazon: Zepto & more.
✔ Scraped Data: Price Insights Top-selling SKUs
"Actowiz's helped us reduce out of stock incidents by 23% within 6 weeks"
✔ Scraped Data, SKU availability, delivery time
With hourly price monitoring, we aligned promotions with competitors, drove 17%
Actionable Blogs, Real Case Studies, and Visual Data Stories -All in One Place
Explore the top 500 trending ecommerce products in the USA for December with real-time data, pricing shifts, and marketplace insights powered by Actowiz Solutions.
A detailed case study on weekly stock-out analysis for top grocery SKUs using Actowiz Solutions’ real-time availability tracking and data intelligence.
Real-time grocery price changes across Walmart, Instacart and Target. Track top SKU drops, increases and hourly volatility with Actowiz Solutions.
A detailed research report analyzing December ecommerce deals, discounts, and price drops across USA and UAE marketplaces powered by Actowiz Solutions.
A detailed blog uncovering assortment gaps across major USA ecommerce marketplaces with real-time insights powered by Actowiz Solutions.
A complete analysis of December price drops and deal patterns across Amazon, Walmart, and Target using real-time ecommerce intelligence from Actowiz Solutions.
Discover how pack-size differences drive major price wars across 10 leading grocery platforms. A data-driven case study powered by Actowiz Solutions.
A case study exploring how the HungerStation Dataset for Restaurant and Order Data enables accurate order forecasting and improved delivery efficiency through data-driven analysis.
Enhance deep learning performance with large-scale image scraping. Build diverse, high-quality training datasets to improve AI accuracy, object detection, and model generalization.
Uncover how data-driven strategies optimize dark store locations, boosting quick commerce efficiency, reducing costs, and improving delivery speed.
A detailed December ecommerce price index report comparing category-wise pricing trends across USA & UAE markets powered by Actowiz Solutions.
Weekly research report on top-selling SKUs across USA marketplaces like Amazon, Walmart, and Target, powered by Actowiz Solutions’ real-time ecommerce intelligence.
Benefit from the ease of collaboration with Actowiz Solutions, as our team is aligned with your preferred time zone, ensuring smooth communication and timely delivery.
Our team focuses on clear, transparent communication to ensure that every project is aligned with your goals and that you’re always informed of progress.
Actowiz Solutions adheres to the highest global standards of development, delivering exceptional solutions that consistently exceed industry expectations