I’ve been wrestling with Safari rendering inconsistencies across macOS and iOS for months. Every time we push a design change, something breaks in unexpected ways—different font rendering, layout shifts, spacing issues that don’t show up on Chrome. It’s like chasing ghosts.
Recently I decided to stop manually writing Playwright scripts and try describing what I actually wanted to test in plain language. The idea was to see if an AI could take something like “test that the hero section renders consistently across Safari on iPhone 12, iPad Pro, and Mac” and turn it into a working cross-browser visual regression workflow.
The workflow it generated was… surprisingly functional. It set up proper viewport sizes, handled the async rendering waits without me having to think about it, and even added some basic visual regression checks I wouldn’t have thought to include. But here’s where I got skeptical: does it actually stay stable when the site gets redesigned? Or does it just work for the first run and then becomes another source of flaky tests?
I’m curious if anyone else has tried this approach with WebKit specifically. When you’re dealing with Safari’s quirks across different OS versions, does an AI-generated workflow hold up over time, or does it need constant tweaking like traditional Playwright scripts?
The stability issue you’re hitting is real, but the approach of describing what you need in plain text is exactly where AI automation shines. Instead of fighting with selector brittleness, you can set up a continuous workflow that regenerates your tests based on your description whenever things change.
What I’ve seen work is building the workflow so the AI regenerates the selectors and visual baselines automatically when a redesign happens. You describe the goal once, and the system adapts. No more manual Playwright rewrites.
The key is setting up the workflow to be autonomous about updates, not one-off. That’s where the flakiness disappears—because you’re not maintaining static scripts, you’re maintaining the description of what should be tested.
I ran into similar problems with Safari rendering across devices. The plain text approach works, but stability depends a lot on how specific your description is. If you’re vague, the AI fills in gaps and makes assumptions that change between runs.
What helped me was being explicit about edge cases—timeouts, network delays, specific viewport dimensions. When I stopped trying to make the description short and comprehensive and started including context about what usually breaks, the generated workflows became much more consistent.
The real gain isn’t that it never needs adjusting. It’s that when you need to adjust, you just update your description, not rewrite the entire test automation.
From what I’ve experienced, AI-generated WebKit workflows stay stable if you treat them like living documentation rather than static code. The workflow itself doesn’t degrade, but your test descriptions need to evolve alongside your product changes.
I started keeping a changelog of what the description includes and why. When a design shift happens, I update that description with the new context. The regenerated workflow picks up on the changes naturally. The workflows I’ve maintained this way have been more stable than my hand-written Playwright tests because the AI is consistent about what it tests, whereas I’d sometimes miss edge cases when updating manually.
The stability question really hinges on whether you’re treating this as a one-time generation or as continuous automation. Generated workflows for WebKit testing do hold up well if the underlying description remains accurate. The key difference from manual scripts is reproducibility—the same description always produces the same assertions.
What tends to break isn’t the workflow itself but when your product changes and the description no longer matches reality. Setting up the workflow to regenerate periodically or on-demand when you push changes prevents that misalignment. I’ve seen teams successfully maintain cross-device Safari tests this way for months without touching the underlying automation.
It’s stable if you keep the description updated. I’ve found regenerating the workflow when major ui changes happen beats rewriting scripts manualy. AI-generated tests are actually more consistent than hand written ones.