Skip to main content

Playwright .toHaveScreenshot() vs Webshot Archive: A Comprehensive Comparison

· 6 min read
Toshi Moto
Front End Engineer

When it comes to visual regression testing, Playwright's .toHaveScreenshot() method has become a popular choice for developers. However, there's a growing need for more sophisticated screenshot management that goes beyond simple file-based storage. This is where Webshot Archive comes in, offering a centralized, team-friendly approach to visual testing.

The Core Difference: Centralized vs Distributed

Playwright's Approach

Playwright stores screenshot files directly in your Git repository alongside your test code. While this keeps everything in one place, it comes with several limitations:

  • Repository Bloat: Screenshots accumulate over time, making your repository larger and slower to clone
  • Limited Collaboration: Screenshots are tied to specific branches and commits, making it hard to share across teams
  • Required Manual Step: Every time a developer checks in code that affects UI, they must remember to run the --update-snapshots command to update baseline screenshots
  • Manual Git Navigation: To view screenshots across history, you must checkout branches one at a time and navigate through commits manually

Webshot Archive's Approach

Webshot Archive provides a centralized, hosted repository for all your visual regression screenshots:

  • Clean Repositories: Keep your code repository focused on code, not binary assets
  • Dedicated Infrastructure: Optimized storage and retrieval for screenshot data
  • Global Accessibility: Team members can access screenshots regardless of their local Git state
  • Easy Historical Viewing: Browse and compare screenshots across commits without checking out different branches

Sharing to Discord, Slack, and more

· One min read
Toshi Moto
Front End Engineer

New feature unlocked! You can now share your projects to Discord (Slack, and email coming soon). Webshot Archive now supports integration with Discord via webhooks. What this means is that developers can now send clips of changes to your projects to Discord channel. This will help teams catch changes, bugs, and features earlier in the development cycle. Discord Webhook

View the example in the Webshot Archive Discord docs channel.

Github Actions

· 2 min read
Toshi Moto
Front End Engineer

Github Actions run on a Github Event, however, the default behavior of the Webshot Archive Github Action will differ based on the type of event being run.

When running a workflow on pull_request, the Webshot Archive Github Action will default its compareCommitSha (the commit to compare against) based on ${{ github.event.pull_request.base.sha }}. For a workflow that runs on push, the compareCommitSha is based on ${{ github.event.before }}. Override this behavior by setting the compareCommitSha in the action input options. See the API docs for more information of customizing the action input options.

note

To comment on a pull request with the captured images, the workflow must run on pull_request.

Image Diffing

· 2 min read
Toshi Moto
Front End Engineer

Understanding diffs are at the core of Webshot Archive. This blog post will explain how diffs work and where they are used.

Diffs in Github Comments on PR

When the Webshot Archive Github Action runs, your screenshots are uploaded to our hosted storage. The server will then create a diff of the new screenshot and the screenshot the action tells it to compare to. This diff is then uploaded to the Webshot Archive storage along with the original screenshot and added to the Github comment on your PR.

Github Comment with Diff

You can also see the diff on the Screenshot Comparison Carousel. In this example the diff is almost 4000 pixels which is a lot but can give some context to what a little value might be. In my opinion 250 pixels is a good starting point. Some pages though may have a lot of noise and a higher value may be needed which can be done on an individual basis (details below).

Screenshot Comparison Carousel