Puppeteer and Playwright can be particularly useful when scraping data accessible only behind a login wall. This article shows a practical example of such a case.
Giovanni Rago Giovanni RagoPuppeteer and Playwright can be used to create PDFs from webpages. This opens up interesting automation scenarios for tasks such as archiving, generating invoices, writing manuals, books and more.
This article introduces this functionality and shows how we can customise the PDF to fit our needs.
Giovanni RagoThe HyperText Transfer Protocol (HTTP)
This article shows how we can use cookies and the Web Storage APIs to set state in our Puppeteer and Playwright scripts, opening up new scenarios and saving on execution time.
Giovanni RagoThe need for fast and responsive applications has never been greater because of the move from desktop to mobile
This article aims at giving a practical introduction to the whys and hows of web performance, without getting lost in the depth or breadth of this massive topic.
Giovanni RagoWith Puppeteer and Playwright, writing scripts by hand is not the only option. Since browser automation first became possible, there have always been tools trying to simplify script creation. Such instruments normally aim to help users with little or no scripting skills use automation tools, while also saving more expert users time when creating brand new scripts.
Recorders can be used to quickly generate code for a scenario, saving time users would otherwise have to spend inspecting the various pages to find valid selectors. When creating multiple scripts, this adds up to a noticeable amount of time saved.
Hannes LenkeWebsites might expose files for users to download and then access from their local machine. Common cases are downloading tickets, receipts and itineraries.
Giovanni RagoWhen we browse the web, a series of HTTP requests and responses are exchanged between our browser and the pages we are visiting. There are scenarios in which it is useful to monitor or manipulate this traffic, instead of letting it happen as-is.
We call the action of extracting data from web pages web scraping. Scraping is useful for a variety of use cases:
- In testing and monitoring, asserting against the state of one or more elements on a page
- In general, gathering data for a variety of different purposes
Giovanni RagoWhile automation tools are fundamental to modern software development, they also have the innate potential to be used for malicious purposes. This applies to Puppeteer and Playwright, too.
Giovanni RagoMost services allow users to manage their account settings. These oftentimes have far-reaching implications on how the user experiences the platform. Verifying that the account settings can be viewed and modified is key in making sure we are offering a smooth service.
Giovanni RagoWe define test data as any data we consistently use to verify a certain property, like functionality or performance, of a system. Another popular term for the same concept is fixture. We will use these interchangeably.
Giovanni RagoHeadless browsers can be leveraged for testing in a variety of ways, and different scenarios do command different approaches. That being said, there are some general pointers most should follow in order to keep their tests valuable.
Here, we define valuable as sustainably expressing meaningful, truthful information about the state of a system. A test that does not reliably fulfill these criteria should be fixed, if possible, or simply removed.
Giovanni RagoSearching a website is an everyday action for most internet users. For most services, the speed at which their customers are able to get to the products they are looking for is directly tied to revenue. To enable that, a performant and reliable search function is needed.
Giovanni RagoWebshops and subscription-based services often offer discounts through coupon codes. Applying a valid coupon code during checkout might reduce the price of one, several, or all items in the shopping cart.
Giovanni RagoAnother key website flow that needs to be closely monitored is any sort of checkout. You want to be 100% sure your users are able to pay for goods or services you might be offering, at any given time.
Giovanni Rago