How to Build a Price Tracker

Step 1: Set Up the Project and Dependencies

Start by creating a Python project directory with a virtual environment. Python is the preferred language for price tracking because of its rich ecosystem of scraping, data processing, and scheduling libraries. You will need four core dependencies to get started.

Playwright is a browser automation library maintained by Microsoft that controls Chromium, Firefox, or WebKit browsers programmatically. Unlike HTTP-only scrapers, Playwright renders JavaScript, handles dynamic content loading, and can interact with page elements like dropdowns and variant selectors. This is essential for modern e-commerce sites where prices are often loaded asynchronously after the initial page render. Install it with pip install playwright followed by playwright install chromium to download the browser binary.

SQLite ships with Python's standard library (via the sqlite3 module), so it requires no additional installation. SQLite is an excellent choice for price trackers because it stores everything in a single file, requires no server configuration, and handles the read/write patterns of a price tracker efficiently. For larger-scale operations (thousands of products with high-frequency checks), PostgreSQL provides better concurrent write performance and more advanced querying capabilities.

APScheduler (Advanced Python Scheduler) manages recurring job execution within your Python application. Install it with pip install apscheduler. Alternatively, you can use system-level scheduling with cron (Linux/macOS) or Task Scheduler (Windows), but APScheduler keeps the scheduling logic within your Python codebase and offers features like job persistence and missed job recovery.

playwright-stealth is an optional but recommended library that patches Playwright's browser fingerprint to reduce detection by anti-bot systems. Many e-commerce sites use bot detection services that identify automated browsers by their JavaScript properties. Stealth mode modifies these properties to more closely resemble a human-operated browser. Install with pip install playwright-stealth.

Step 2: Build the Scraping Layer

The scraper is the component that visits product pages and extracts pricing data. A well-structured scraper separates the browser management, page navigation, and data extraction into distinct functions so that adding new target sites is straightforward.

The basic scraping flow is: launch a headless browser instance, navigate to the product URL, wait for the price element to render, extract the price text and any additional data (product name, availability, seller), clean and normalize the extracted values, then close the browser. Using Playwright's async API improves performance when checking multiple products because you can run several browser contexts concurrently rather than processing pages sequentially.

Price extraction requires identifying the correct CSS selector or XPath for the price element on each target website. Inspect the product page in your browser's developer tools to find the element that contains the price. For Amazon, the price is typically in an element with the id priceblock_ourprice or within a span.a-price-whole element. For other retailers, the selectors vary by site. Store your selectors in a configuration file or database so they can be updated without modifying the scraper code when a site changes its layout.

Handle common extraction challenges by building defensive logic into your scraper. Some products display a price range instead of a single price, which requires parsing both the low and high values. Out-of-stock products may not show a price at all, so your scraper should detect availability status and handle missing prices gracefully. Products with multiple variants (size, color, configuration) may show different prices depending on the selected variant, requiring your scraper to either extract the default price or iterate through variants.

Normalize extracted prices by removing currency symbols, thousands separators, and whitespace, then converting to a numeric type. Store prices as integers representing cents (or the smallest currency unit) rather than floating-point numbers to avoid rounding errors in calculations and comparisons. Record the currency alongside the price if you monitor products across multiple countries.

Step 3: Design the Database Schema

The database stores your product catalog, price history, and alert configuration. A minimal schema needs three tables: products, price_snapshots, and alerts.

The products table stores metadata about each tracked product: a unique identifier, the product name, the target URL to scrape, the CSS selector for the price element, the competitor or retailer name, and any categorization fields useful for analysis. This table serves as the master list of what your tracker monitors. Adding a new product to track means inserting a row here with the URL and selector.

The price_snapshots table stores every price observation: a foreign key to the products table, the observed price (as an integer in cents), the timestamp of the observation, and optional fields for availability status, seller name, and shipping cost. This is the core data asset of your tracker. Every scheduled run adds one row per product to this table, building up the historical record that enables trend analysis.

The alerts table defines notification rules: which product to watch, the threshold condition (price below a target, price changed by more than a percentage, product back in stock), the notification method (email, Slack webhook, SMS), and a flag indicating whether the alert has already fired for the current condition. The alert table prevents duplicate notifications by tracking which alerts have been triggered and resetting them when the condition clears.

Index the price_snapshots table on (product_id, timestamp) for efficient time-series queries. As the table grows, you may want to add a retention policy that archives or deletes snapshots older than a certain age, though for most price trackers the data volume remains manageable even after years of operation.

Step 4: Add Change Detection and Alerts

Change detection compares each new price snapshot against the previous snapshot for the same product and identifies meaningful movements. The simplest approach queries the two most recent snapshots for each product and calculates the difference.

Define what constitutes a "meaningful" change to avoid alert fatigue. Many products experience minor price fluctuations (a few cents or fractions of a percent) due to rounding, currency conversion, or platform-specific pricing rules. A good default threshold is to only flag changes greater than 1-2% or above a minimum absolute amount. You can set different thresholds per product or category based on how price-sensitive each market segment is.

For alert delivery, email is the most universal method. Python's smtplib module sends email through any SMTP server, including Gmail, SendGrid, or Amazon SES. For team notifications, a Slack or Microsoft Teams webhook provides instant visibility in a shared channel. Format alert messages to include the product name, old price, new price, percentage change, and a direct link to the product page so the recipient can verify and act immediately.

Consider implementing tiered alerting based on the magnitude and direction of price changes. A small decrease might generate a log entry but no notification. A significant decrease on a tracked competitor product might send an email to the pricing team. A large sudden increase might indicate a data quality issue (the scraper hit the wrong element) and should be flagged for review rather than treated as a real price change.

Step 5: Schedule Recurring Checks

A price tracker only delivers value if it runs automatically on a regular schedule. The scheduling component triggers the scraping pipeline at defined intervals and handles operational concerns like error recovery and resource management.

Using APScheduler, you can define a job that runs your scraping function at fixed intervals (every hour, every 6 hours, daily at a specific time). APScheduler supports interval triggers, cron-style triggers for specific times, and date triggers for one-time execution. For most price trackers, an interval trigger running every few hours provides a good balance between data freshness and server load on target websites.

Add jitter (random delay) to your schedule to avoid hitting target websites at exactly the same time on every run. A predictable scraping pattern is easier for anti-bot systems to detect and block. Adding a random delay of 1-5 minutes to each scheduled run makes your access pattern less distinguishable from organic traffic. Similarly, randomize the order in which products are checked and add variable delays between individual page loads.

Proxy rotation is essential for sustained scraping at any significant scale. If all requests come from a single IP address, target websites will eventually rate-limit or block that IP. Residential proxy services provide pools of IP addresses that rotate with each request, making your scraping traffic appear to come from many different users. For smaller operations (under 100 products with daily checks), a small set of datacenter proxies may be sufficient. For larger operations, residential proxies provide better reliability.

Build error handling into the scheduler to manage scraping failures gracefully. Individual product scrapes can fail because of network timeouts, page structure changes, CAPTCHAs, or temporary site outages. Log failures for review but do not let a single failure stop the entire run. Implement a retry mechanism that attempts failed products again after a delay, and alert if a product fails consistently across multiple runs, which likely indicates a broken selector that needs manual repair.

Step 6: Add Reporting and Analysis

Once your tracker has accumulated price history, the analysis layer transforms raw data into actionable insights. Start with a few core reports and expand as your needs evolve.

A current competitive position report shows the latest price for each product alongside competitor prices, ranked by price. This is the daily reference that pricing teams use to identify products where you are priced above or below the market. Include the price difference (absolute and percentage) relative to the cheapest and most expensive competitors.

A price change log lists all detected price changes over a selected time period, sorted by recency or magnitude. This report surfaces the most significant competitor movements and helps pricing teams prioritize which changes warrant a response. Filter options for specific competitors, product categories, or price change direction (increases vs. decreases) make the log more actionable.

A trend visualization plots price history over time for individual products or product groups. Line charts showing your price alongside competitor prices over weeks or months reveal patterns invisible in snapshot data: gradual price erosion, seasonal cycles, promotional cadences, and long-term pricing trajectory shifts. Python libraries like Matplotlib or Plotly generate these charts, which can be embedded in reports or served through a simple web dashboard.

For teams that prefer dashboards over static reports, a lightweight web framework like Flask or FastAPI can serve a monitoring dashboard that queries the SQLite database and renders current prices, alerts, and trend charts in a browser. This adds development complexity but provides a more interactive experience for daily pricing review.