Ad verification

You run or verify ad campaigns and need to see which landing page a visitor in a given region on a given desktop reaches, and through which redirects. This walkthrough opens an ad link once per region and persona, prints every main-frame response on the way and saves a screenshot of the page it ends on. Opening a tracking link can register a click with the ad network. Check links from campaigns you run or are engaged to verify.

What Apostate changes

A persona is the platform a launch presents. It sets the operating system in the User-Agent, the Client Hints and navigator.platform, which ad servers and landing pages use to pick a creative or a download page. Personas are Windows, macOS and Linux desktops. Apostate has no mobile personas. A proxy gives the visit an IP address in the region. Before launch, the package sets the locale and timezone from the proxy’s exit, so the language and timezone agree with the IP address.

Set up

pip install apostate
apostate install
apostate fonts install windows
git clone https://github.com/heretic-tech/apostate.git
cd apostate/examples/use-cases/ad-verification

apostate fonts install windows is for Linux and macOS hosts (Fonts). verify_ads.py takes an ad or tracking link. For each region, it reads a proxy from APOSTATE_PROXY_US, APOSTATE_PROXY_DE or APOSTATE_PROXY_FR and skips a region whose variable is not set. With no link, it runs against a local ad server. The server redirects /click to /track, and /track to a landing page in the language of Accept-Language. The landing page’s script sends Mac visitors on to a /mac page.

The script

follow() opens the link with one persona and records each response to a main-frame navigation. That covers HTTP redirects and navigations a page’s script starts:

def follow(url, persona, options, screenshot):
    """Open url and return the main-frame responses in order, the final URL and the page's region."""
    chain = []
    with launch(fingerprint=42, fingerprint_platform=persona, **options) as browser:
        page = browser.new_page()
        page.on("response", lambda response: chain.append((response.status, response.url))
                if response.request.is_navigation_request() and response.frame == page.main_frame
                else None)
        page.goto(url)
        page.wait_for_load_state("networkidle")
        region = page.evaluate("[Intl.DateTimeFormat().resolvedOptions().timeZone, navigator.language]")
        page.screenshot(path=screenshot, full_page=True)
        return chain, page.url, region

Each region’s launch options come from its proxy. In the local demo, a region is only its locale and timezone:

    for region, locale, timezone, variable in REGIONS:
        proxy = os.environ.get(variable)
        if server is not None:
            # The local demo has no proxies, so each region is its locale and timezone.
            options = {"locale": locale, "timezone": timezone, "geoip": False}
        elif proxy:
            # Through a proxy the package sets locale and timezone from the exit.
            options = {"proxy": proxy}
        else:
            print(f"{region}  skipped: {variable} is not set")
            continue

Run the demo

python3 verify_ads.py

US  windows  America/New_York  en-US
    302  http://127.0.0.1:8766/click?campaign=spring
    302  http://127.0.0.1:8766/track?campaign=spring
    200  http://127.0.0.1:8766/landing/en?campaign=spring
    final http://127.0.0.1:8766/landing/en?campaign=spring
    screenshot out/US-windows.png
US  macos  America/New_York  en-US
    302  http://127.0.0.1:8766/click?campaign=spring
    302  http://127.0.0.1:8766/track?campaign=spring
    200  http://127.0.0.1:8766/landing/en?campaign=spring
    200  http://127.0.0.1:8766/landing/en/mac?campaign=spring
    final http://127.0.0.1:8766/landing/en/mac?campaign=spring
    screenshot out/US-macos.png
DE  windows  Europe/Berlin  de-DE
    302  http://127.0.0.1:8766/click?campaign=spring
    302  http://127.0.0.1:8766/track?campaign=spring
    200  http://127.0.0.1:8766/landing/de?campaign=spring
    final http://127.0.0.1:8766/landing/de?campaign=spring
    screenshot out/DE-windows.png
DE  macos  Europe/Berlin  de-DE
    302  http://127.0.0.1:8766/click?campaign=spring
    302  http://127.0.0.1:8766/track?campaign=spring
    200  http://127.0.0.1:8766/landing/de?campaign=spring
    200  http://127.0.0.1:8766/landing/de/mac?campaign=spring
    final http://127.0.0.1:8766/landing/de/mac?campaign=spring
    screenshot out/DE-macos.png
FR  windows  Europe/Paris  fr-FR
    302  http://127.0.0.1:8766/click?campaign=spring
    302  http://127.0.0.1:8766/track?campaign=spring
    200  http://127.0.0.1:8766/landing/fr?campaign=spring
    final http://127.0.0.1:8766/landing/fr?campaign=spring
    screenshot out/FR-windows.png
FR  macos  Europe/Paris  fr-FR
    302  http://127.0.0.1:8766/click?campaign=spring
    302  http://127.0.0.1:8766/track?campaign=spring
    200  http://127.0.0.1:8766/landing/fr?campaign=spring
    200  http://127.0.0.1:8766/landing/fr/mac?campaign=spring
    final http://127.0.0.1:8766/landing/fr/mac?campaign=spring
    screenshot out/FR-macos.png

The first line of each block is the region, the persona, and the timezone and language the landing page read. The two 302 lines are the HTTP redirects. The tracker chose the landing page from Accept-Language. Under the macOS persona, the landing page’s script moved on to /mac, which shows as a second 200.

Run through regional proxies

Set a proxy for each region you want to check, then pass the link:

export APOSTATE_PROXY_US="socks5://user:pass@proxy.example:1080"
export APOSTATE_PROXY_DE="socks5://user:pass@proxy.example:1081"
python3 verify_ads.py "https://ads.example/click?id=123"

The script skips a region with no proxy. With no variable set, the output is:

US  skipped: APOSTATE_PROXY_US is not set
DE  skipped: APOSTATE_PROXY_DE is not set
FR  skipped: APOSTATE_PROXY_FR is not set

Through a proxy, the package looks up the exit’s location before launch, so the timezone and language on each block’s first line are the ones it found. Check that they belong to the region. If the lookup fails, the package prints a warning that starts with apostate: , and the persona uses en-US and the host’s timezone. From 0.4.4 the warning is also in browser.apostate_diagnostics["warnings"]. Proxies covers the lookup.

Points for this job

Proxies. Use one exit per region, sticky for the length of a launch. Proxies covers sticky sessions.
Seeds. Each persona uses seed 42, so a rerun visits with the same two machines and a change in the result comes from the campaign. To see how a campaign treats other machines of one persona, add seeds.
What the chain records. The chain holds HTTP redirects and the navigations a page’s script starts, such as location.replace(). An ad inside an iframe navigates its own frame, so filter on that frame instead of page.main_frame. wait_for_load_state("networkidle") returns after 500 ms with no network connections. The chain misses a redirect that a page schedules later than that. To catch it, add page.wait_for_timeout() before the screenshot.
Headless. The screenshots are full-page captures from a headless launch, which presents the persona’s screen and window.

Examples

Use cases

What Apostate changes

Set up

The script

Run the demo

Run through regional proxies

Points for this job

​What Apostate changes

​Set up

​The script

​Run the demo

​Run through regional proxies

​Points for this job

What Apostate changes

Set up

The script

Run the demo

Run through regional proxies

Points for this job