Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Empower Apps

Empower Apps artwork

Swift Testing with Rachel Brindle

Jul 18, 202539 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Rachel Brindle, maintainer of Nimble and Quick, comes on to talk about what's new in Swift Testing and what the future of Swift Testing may hold. Guest Rachel Brindle Rachel Brindle (@younata@hachyderm.io) - Hachyderm.io younata (Rachel Brindle) Rachel Brindle (@rachelbrindle) • Instagram photos and videos Related Links What's new in Testing, 2025 Edition nalexn/ViewInspector: Runtime introspection and unit testing of SwiftUI views Quick/Nimble: A Matcher Framework for Swift and Objective-C Quick/Quick: The Swift (and Objective-C) testing framework. pointfreeco/swift-snapshot-testing: 📸 Delightful Swift snapshot testing. swift-evolution/proposals/testing/NNNN-polling-confirmations.md at younata/testing-polling-expectations · younata/swift-evolution Related Episodes Live from CommunityKit WWDC 2025 with Matt Massicotte The Case of the Crimson Test Suite with Daniel Steinberg Empowering Accessibility with Via Fairchild It Depends with Brandon Williams Practical Year - Part 1 with Donny Wals SwiftUI Tips and Tricks with Craig Clayton Mobile System Design with Tjeerd in 't Veen Social Media LinkedIn - @leogdion GitHub - @brightdigit GitHub - @leogdion TikTok - @brightdigit Mastodon - @leogdion@c.im Youtube - @brightdigit BlueSky - ‪@leogdion.bsky.social‬ Twitter Leo - @leogdion Twitter BrightDigit - @brightdigit Credits Music from https://filmmusic.io "Blippy Trance" by Kevin MacLeod (https://incompetech.com) License: CC BY (http://creativecommons.org/licenses/by/4.0/) (00:00) - Comparing Swift Testing and XCTest (11:08) - SwiftUI Testing (17:37) - Advanced Testing Techniques and Tips (23:47) - Future of Swift Testing Thanks to our monthly supporters Steven Lipton ★ Support this podcast on Patreon ★

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Welcome to another episode of Empower Apps. I'm your host, Leo Dion. Today I'm joined by Rachel Brindle. Rachel, thank you so much for taking time to come on Yeah, before we get into Swift Testing and all the new stuff with testing. I'll let you go ahead and introduce yourself. Yeah, so I am Rachel. I am a principal engineer at Autodesk. And I have been involved with testing, test-driven development and such, basically my entire career. A lot of fairly deep knowledge of like how to go through and test things, as

well as mentoring others in learning how to test code, how to write better tests that aren't brittle and won't break all the time and so forth. Awesome. So what year did Swift Testing come out? That was, It shipped in Xcode 16 last year, but it was open sourced in September, 2023. Okay. So you did a lot of work with XCTesting. You have a few libraries you actually built, for XCTesting, to help folks. What did you think about the process as far as Swift Testing is concerned

and how we got to where we are now? How we went from XCTesting to Swift Testing. And what do you miss? what are you happy that we got rid of Things like that, So Swift Testing is obviously, 'cause it's only like publicly, it's only like a year and a half old, though I'm, I know for a fact that like, it didn't just pop outta nowhere. they've been working on it since at least macros were a thing. Swift Testing, if it's still like no more than, it can't be

older than like three years. It is. Nowhere near as mature as XCTest is. XCTest has been, was introduced in iOS seven, iOS eight era. But even still it dates back to much older Objective C focused testing APIs like syn testing kit and so forth. And those date back all the way to the nineties. I was gonna ask about that. 'cause like XCTest, I would assume. Like you said is from quite a while since Objective C. 'cause I can't even imagine

what they used for testing before that. Yeah, it was, I don't think Apple had, so it was initially started as some, third party, like predating. It might even predate like Apple buying next. I don't that. That I, I don't quite know the history there. But it was, there was a send testing kit and like a bunch of like other third party stuff is what people used. At some point in the two thousands, apple started shipping or like including

send testing kit with, Xcode so that you didn't have to download that, And they started to just refine that over time and eventually they rewrote it with XC test and, and shipped that. Okay, so Swift tests, like the big thing I was gonna talk about is what you miss from XCTesting, but then like a part of that is just what we could do with Objective C that we can't get away with in Swift. So there is the open source XCTest.

it's Swift CoreLibs XCTest and that is written entirely in Swift and that enables that has been around since Swift package manager was open sourced back in like 2015 or so. And that is not feature complete with like the private XCTest from Apple. But it does a lot of what you can do with XCTest. XCTest is really much better for building testing tools on top of, like you can dynamically create and register tests at runtime and XC test, which is not a

thing you can do in Swift Testing at all. I know that there are people on the Swift Testing team interested in enabling something like that, but it's just not there yet. Though reading between the lines, they're definitely moving to enable that feature in, Swift Testing. And then there's other minor stuff like, performance tests. The XCTest has the measure API, so you can Figure out how long it takes to run a bit of code so you can see like, oh, do I have any performance

regressions in this area and such? And you can kind of do that with Swift Testing, but for a bunch of reasons, Swift Testing's, concurrent test runner being a major issue there. it's not great. And of course there's not really any equivalent to the measure API. So what did you really like about Swift Testing and where it's headed right now compared to XCTesting? So the thing that I really like about it first, the DSL of the experience of actually writing tests is much nicer than with XCTest.

The macro or just being able to specify like the attest macro and then just like a bear function, is great, especially in Swift six two. I think it's se 4 51 ads just like you can, I think it's bear function. You can use a backtick and then put an arbitrary string or nearly arbitrary string between back ticks and like that's the name of your function. So you can have spaces commas and punctuation in there. that will render nicely as a full human readable string so you don't have to

do this weird thing where like you have attest and then whatever human readable string and then a non-human readable, but like valid function identifier afterwards. Okay, so now you could just make the function whatever you want, and put whatever characters you yeah. That's crazy. That's useful for non-testing stuff, but mostly it's useful for testing. That's like reading the proposal for that. That's like the main motivation they cite. Yeah. Yeah. So yeah, like the, going back to other things that I really like about, Swift

Testing I like how it's much more open. Both like it's developed mostly in the open. there is a testing work group, which I'm a member of. And it's. It, follows a form of the Swift evolution process. Like testing proposals are in the Swift evolution repo. But the proposals are formatted slightly differently and such so I'm really liking that there's a lot more community engagement in the development of Swift Testing. Yeah.

And I think that's the biggest advantage is the fact that it's open source and that we can build a whole set of tools around that and help it facilitate its growth, what point do you think it'd be important for a team to migrate from XCTesting to Swift Testing? Really that depends on the team, there's a ton of tests that are written right now that cannot be easily migrated or like can't, it's not gonna be an easy process to migrate it from XCTest to Swift Testing. Do you have an example? So like anything that accesses like, or modify some like shared global state,

For like one specific test, you might change it to some value. And then another test wants it to be some other value. because Swift Testing runs everything concurrently in process, then you're gonna run into issues if those two tests are ever actually like running at the same time. Should you be doing that in the first I mean no, Okay. Yeah. Yeah. In a way it forces you to be cleaner. Like you'll end up replacing that type of thing with a task local or whatever. And that also brings along all the niceties of like, you're now going to force your tests to actually like build under the Swift six

language mode and all that such. But there are plenty of tests that exist right now that work perfectly fine in the world with XCTest where it's only one test is running at a time in the process. That are going to be difficult to migrate to Swift Testing. Okay. And then the other benefit is it'll be slightly easier to read stuff. You'll get faster runtime of tests because they can execute in parallel but you're gonna get slightly slower build times because macros take a bit of time.

Even tool chain macros, they just add a little bit more time to the compile time. automation tests still aren't a thing and so forth. But yeah, if you have working XC tests, there's no reason to, unless you really want to, there's no reason to go rewrite them into Swift Testing. Maybe. So like that's the approach to that idea is new components are written in Swift Testing. But if it's an existing component that already has tests written for it, then I'm just gonna keep it in XC test.

Do you think macros have a place in filling the gap when it comes to what's missing? In Swift Testing? Yes and no. So well for I guess to correctly answer, yes, they have a gap. But like they are very painful to write and maintain as I'm sure you don't need to be reminded, I looked at how, the expect and require macros and Swift Testing are implemented and like mad respect to the people on the testing team who wrote them, because I do not know if I could.

And there's also the other stuff of like, until very recently you had to rebuild sw, Swift, Swift syntax for like. Third party macros and such, that isn't really a thing now that we have like the cast with Swift syntax as of like 16 four or code 16 four. But like there are, there actually are places for macros. Every so often someone will post in the Swift forums, the community showcase about a library that uses macros to generate test doubles.

Conforming to the protocol specified. So a test double is just kind of a stand in for the real thing. Okay. So you might have your type that has some methods or whatever and you have a protocol that defines that type. You might then use one of these tools to. Create a test double that automatically conforms to and implements all of these methods such that you can then either just directly pass it in and it'll work in the thing. Or you can modify behavior or you could even examine how

it was called and so forth. that's really cool to see and that saves a bunch of boilerplate. Yeah, so that's one of the main examples of using macros to fill in the gap and do things that Apple probably isn't gonna do, at least not with Swift Testing. At least not in the next decade. Yeah, you had this like marketable macro, which is really cool. I just started looking at that, because like that's one of the things people love about Objective C is everything is dynamic so you can go in and swizzle

and do all sorts of crazy stuff, whereas Swift is super strict about that stuff. Like for good reason, like, outside of using tests to like mock out behavior, we might get the into this in a, in a little bit, but like this is part of what made testing UI kit a lot less painful was being able to swizzle out a whole bunch of stuff. Which you can't even do any of that with Swift test with Swift UI. And like Swift UI is also not untestable. Yeah. Well, let's talk about that. That's one of the biggest complaints is how SwiftUI is testable.

What does that mean exactly? Yeah, so you can write tests for Swift UI using Swift UI but you can only write automation tests for it. So that's like the XCUI test stuff that, works by spinning up your app in the simulator and then having another process that actually runs the tests that. Uses IPC to interact with the app using the accessibility system. Yep. and out of the box you can only like, just interact with the

UI and examine stuff in the ui. You can't like go in and examine oh, did this insert something into, did this actually insert something into the database or is the UI just lying to me? Out of the box, it's going to make like every single network request and so forth, which it's not really. But there's a bunch of third party stuff to make, automation tests, or like to fake out network requests and so forth and like enable you to better inspect stuff going on at automation tests. Yeah, well, like one thing I hear a lot about is snapshot testing as a way, which

I think is from the point free team. so there are several ways you can do snapshot testing. There's an, older, API from I think Uber was the first who made it iOS snapshot test case both of these, the 0.31 does a little bit more, you have a little bit more options to actually inspect it. it works by just rendering the UI into an image and then basically working entirely as a regression test of like, you render the eye into an image. You test that and you just assert that. The image is the same as a previously recorded image

Yep. That can work either as like, like an XCUI automation test or as more of a unit test, like in process test. And that's fine. Like it works for like pure regression stuff. There's, it's very brittle. Like I do not envy the people with, with snapshot tests, going into who are probably not having a very fun summer, having to rerecord all of their stuff for like the liquid glass design, language change.

And especially like each new beta, it's gonna change. But even going between like minor versions of the os, there, there are like gonna be slight changes to how things render and so forth. There's even like stuff of whether when the simulator runs on like an Intel or like a, apple, silicon Mac you, it will render it slightly different. So you have to do like some sort of like fuzzy image compare instead of just doing a straight up, You're right. pixel by pixel compare. Okay. so like snapshot tests work, but they're extremely brittle.

When I say that Swift UI isn't testable, what I mean is that I can't write a unit test for it. One of the things that you can do with UI kit is you can directly call the methods that Youi kit itself calls when it's like when you tap for that lead to a button callback being called. So it's possible for me to write a test that's just like, Hey, when this button is tapped I write to the database and I can actually then inspect that the database was written to, or that a test double that represents the database was talked to.

and that's not something you can do with Swift UI. You can't simulate a button. Press in test. could you use something like an inspect and push the button? Yes, there is a third party library called a view Inspector. I'll add it to your, show notes To its credit. It does a really good job of allowing you to inspect and test like, Swift UI stuff, Swift UI views and so forth. But it's incomplete. there's a bunch of APIs from Apple that it doesn't have good inspection for. it uses the mirroring API to be able to inspect things.

And then it has to know about specific. Like sub components within to, or like to go like inspect, oh, this was the, UI button or the, sorry, the Swift UI button. tap handler and such. It doesn't automatically support new Apple provided, Swift UI views by default. It does work with like your own third party views 'cause. But it works. It's also brittle, it's vulnerable to internal changes to in Swift UI of That's exactly what I was gonna say

that happened last year. So, yeah. really what I would love to see is first party support for that. I know for a fact that every time, 'cause I bring it up every year and a WW TC lab that both the Swift UI and testing teams get that very frequently as a request. But for various reasons they haven't been able to deliver that. How would you picture something like that? Like you just create a Swift to eye view and then some sort of

select, like how does web do it? Right? Web would just be like, oh, grab the button using a selector, and then push. Yeah. So that's exactly what I'm thinking. Like, something that I've been working on as like a blog post is, exploring what that might look like. Through the lens of like, this is what we might try to do for UI kit and then because we can do it for UI kit. I like the approach, the interacting through the accessibility system that automation tests take.

'Cause that already exists, that forces you to actually add accessibility, support, I was gonna say that. And such. I think that'd be a really good thing to do. plus it's also fairly, like black box, like you could call in an automation test, you can call a tap on an element that cannot be tapped. It's non-interactive and nothing will happen. Like you won't get a compile error. It's just that nothing will happen and your test will silently fail. or maybe it won't fail depending on what your next assert is that's something that I think is.

Really powerful as a direction to go maybe not exactly mirroring the, public API of automation tests. But that's a great starting point of let's interact through the accessibility system to be able to like pull on, pull an element that should exist. and then like be able to interact with it, test or tap on it, swipe on it, and so forth. Yeah, that makes complete sense. So I wanna jump back, talk a little bit more about Swift Testing. So one of the things I really liked about it is the way that you can organize

your tests, without needing to do certain things with classes and stuff. Kind of explain the whole suite testing thing and what are some things developers should be thinking about when they write their tests and how they organize them. Yeah, one of the really cool things about Swift Testing is the fact that you can nest sweets as much as you want. you can have zero sweets, so just bare functions that are with that test macro. You can organize, you can have like a single layer of sweets.

You can have as many as you want. There is, you do kind of run into the issue where, if you have properties on like the parent suite, a child suite cannot reference it because they're structs and they can't know about, they don't, they're not getting like an a reference to like an instance of the parent struct and so forth. So you can't you have that limitation. I'm hoping that once apple. Provides like more APIs for like dynamically adding tests and so

forth, dynamically that there'll be community solutions to that. Something like what the quick framework provides for XCTest. Could you give like a use case of that? so a parent suite info or metadata. Yeah, yeah. So that, that's something like, Even just as simple as like having shared setups. So you only have to create the object being tested like in one place in the, like in it for the, in like the parent or whatever, the outermost suite. And then later on, like you might have like a child suite that is just calling a

method on that object that's being tested. That's. One way to do that. I also like to nest suites, or I would like to nest suites for like, conditional and such, a child, one suite for if this conditional is true. Another for if it's false and such. Because then you also get like the nice fo code, code voting capabilities well, I don't really care about the positive case right now, so I'm just gonna fold that up and only look at the negative case. Like what,

what I use a lot, and we'll get into traits later, but I have to have, I'll have a suite that I do a lot of full stacks with stuff and so I'll have something that's only supported on Apple platform, so I end up having to do like pound if can import or something like that to kind of distinguish between the two SOI, for instance. And then I have to create a variable and then use that as a global variable that says whether it's available or not. which it's kind of a mess, but it works.

We'll talk about traits more but back to like sweets. What do you think, what are some tips you have as far as how you should organize your tests? Yeah, I personally tend to just do one layer of sweets just because I would like to nest them for conditionals, but doesn't really work out well. So I just do one layer and then either organizing them with tags or using. good old Mark comments. Like I usually just do, I'm still stuck in the way of like. I'll have a test suite lined up with a type essentially and do it that way.

Is that, do you feel like that's still a good way to go about that's how I do it. I'll have like a suite that's like, my object tests Okay. I'll And it's good for like code coverage too, because then you could tell, okay, what is this testing Yeah. But you could also do that, without the suite. It does make it nicer in terms of getting the, test diamond, on the left side of the editor because then you can just select everything there. But if you didn't want that, You could just have them all on top of the file and then have the file be your top level organization for it,

Got it. but then you don't get like shared setup and so forth. Right, exactly. What are some other traits that people should be aware of? I would recommend making like a custom trait that automatically does the, like, oh, if Mac, if iOS, if Linux or whatever, How hard is it to do that? it's not that hard. you can, create a type that conforms to, you'll probably want to do both test trait and sweet trait. and if you do conform to sweet trait, You should implement the is recursive

and then just have that be true is recursive means that for a sweet trait, it also applies that trait to every single like child of it, all the tests, all the sub suites and so forth. Yeah, that makes total sense. you can just have it say like, this also is a disabled trait, or like, this marks the test as disabled. A fun project for me yeah, but it shouldn't take that long to do. Okay. There are the, comment in bug traits, which just allow you to associate like text information.

With a tester suite. Okay. There is the serialized trait which forces the test covered by that suite or the like parameterized test to run serially, so not concurrently, Okay. That's kind of weird. 'cause that only applies to like those specific tests. if you have like two like sibling tests that both have like the serialized trait and you run them, but like the parents, the containing suite isn't serialized. If you run those, then like those two tests will actually run concurrently

or like in parallel with each other. Should you be doing serialized tests? it's a thing that exists. Especially in a post Swift six world, like I haven't run into like a specific use case that needs it. I am a hundred percent certain that that use case exists, otherwise it wouldn't exist. right. But like every time that I've used that I've reached for it, it turns out that actually what I should have just done is made whatever the thing that was forcing the thing to be serial to actually make it.

Be thread safe, like change a global variable into a task local or something like that so I did want to cover your article, which I'll put in the show notes, but what are some highlights about, what's new this year that folks should really start looking at using in their tests? Okay, so the major new change is exit tests, which doesn't apply to iOS. on the server side team? Yeah, service side, even Mac development windows and such.

They allow you to verify that. Code exits properly as you expect code. returns an exit code. It throws Okay. or something like that. So you can basically say, Hey, make sure the signal is the right integer. Yeah, so you standard error context. What? How do you check that? so, The result has STA standard error content as a property. the example I wrote could have been better, Now. 'cause I didn't understand what result was and I was like, I'm

looking at this, and I'm like, okay. So result is basically some sort of structure, which has all that metadata Yeah, so okay. it has the, if you tell it, if you tell your exit test to grab the standard error, then it will. Then that, standard error content will be non nil. If you specify with standard output, it will also be a non nil thing. Those are just an array of, I think it's U eight but an array of, yeah, it's array of U eight. It's array of bites that if you wanna, you then have to like convert it to a

string using one of the methods there. If you wanna do a string comparison. there are like some weird beta bugs in there right now where it will like, capture a lot more of the like standard error than you need. And that's just kind of how the, posik spawn, API Oh, really? So yeah, it's. But it works really well. I'm really impressed with how this works. you can't do this with XC test at all.

Like Apple's never added support for this. So if you wanted to verify that your function had a precondition or such in it, and that precondition worked as you expected, you had to go way outta your way to like. Construct something to do that, which no one has ever done in their life. Yeah so it's really nice to have this. that sounds like a server side thing. Yeah, so really, I think I want it to come to iOS. the sandbox in iOS is a thing and that's gonna be difficult.

what's the use case for iOS? Because like if the, the app shouldn't crash, I mean, it should just never crash. Right. But you do have APIs where it's like, Hey, I expect, for example, array subscript, right? So, or like subscript an array. If you pass negative one then, or if you pass like an out of bounds index, then fatal error, a fatal error. And like that. Whether or not it should or, or should return nil or do or like throw an error or something that you can actually handle is another question.

But that sort of thing exists and we should be able to like handle that test case just to have like that coverage that it's working. I was about to ask that. 'cause a thousand years ago when I did a, an episode with on this very subject, I was asking like, how do you test for fatal error or precondition or assert or any of that stuff? And the way we've had to do it is basically have a way to like overload it essentially. But we don't have anything like that. And this, which what it sounds like is this would work in that case, but then

at the same time it wouldn't work on iOS. Is that correct? Yeah, so the way that I would approach this is I would, I would either just not, or I would like put in like an inject, like a, an injectable using like dependency, injecting like a shim that just in actuality it calls, as you were just saying, like it just calls precondition and actuality. And then in test we can examine that it was called that. For this I might either write, like if it's a cross platform app, like I wanna have it run on Mac OS and iOS then I would just have the.

Have it be covered under the test for macOS with like a pound if or like if os macOS stuff covering that. There's a few things that are promising that isn't in there yet. The thing that I didn't write that I'm most excited for is a thing called, issue handling traits. And these allow you to filter or modify issues. So like if you have a pound expect that fails, like it. Pound expect true, equal, equal false.

That's, guaranteed failure there. Then it allows you to, specify a trait that allows you to modify the issue. You might wanna convert the, severity of it. you might wanna just entirely Yeah. Okay. That's really powerful. I don't think it's gonna happen in Swiss six two. But you think it's possible in the next six Six probably six three. It's actively being discussed. Yeah, there's other stuff there that's, exciting, from what it enables, like,

range confirmations, which were merged in, Swift six one and have been out since Xcode 16, three earlier this year. This is a thing that they view as necessary for automation tests. So range confirmations is basically just, you expect the callback to be called within some amount of range. And the example given in the proposal is using an automation test of like, oh, you have a button and. You can expect it to be tapped, clicked once, but what if the user

actually interacts with the app at that point, like a human operator on the simulator or something like that. Like in that case, you'll get a Furious tap event and it'll be or you'll get like a tap event that the test wasn't expecting and then you'll get a failure that you don't really care for. There's also, in that same vein, attachments, were added, I'm just jumping around in this article. So is, is attachments like any sort of like metadata you want to attach to a test? Yeah. So this will be, this is already pretty useful to have as like.

I can attach like a generated JSON payload in the test. And then for debugging reasons, I want to inspect what this JSON was so that I can more easily see like, oh, this is actually an issue with my production code, or this is an issue with the test, or so forth. But. They're also really useful in automation tests where they're used for storing screenshots of what the app was doing, at the time, or even videos and such. I don't remember if the current code does this, but I know that

Xcode 26 supports recording video Like failing automation tests, which is really cool. Yeah. That's a thing that is probably not necessary, but makes the experience of debugging automation tests much easier, especially when these things are running on ci. both of those are things that I'm really excited for in terms of what they enable. Let's talk about it. Actually before we get into that, you wanna talk a little bit more about specifically your proposal regarding polling confirmations?

Yeah. one of the libraries that I maintain is called Nimble. Nimble is a library that I believe, Swift Testing took a lot of, or at least the pound expect and pound require. APIs took a lot of inspiration from, I haven't asked to confirm this but that provides a, syntax for, it's also inspired by ARS Spec. So, or Nibble is inspired by ARS Spec. So like there's that too. It provides just a syntax for that's better than the XCT assert, in my opinion.

For comparing v. Asserting that some behavior happened, something happened. one of the features of Nimble is something that I call pulling expectations, which is like the form is, expect something to eventually something else. So the idea of a polling expectation is that it will run the thing that you expect. the value being sent into expect is actually a closure. So it'll run that closure many times pulling it continuously until it passes

the matcher, in this case, equal two. So in this case, or in the example I gave, that would always fail, but if you gave it like a method and then that method might change, how its return value is over time then that might eventually pass. This is something that like I think is really necessary for Swift Testing, to support. You can. Kind of do it as a third party, but as I found it actually implementing this as a pitch there are problems with the more naive solution or approach to this.

So yeah, in the article I give an example of like, all right, you call some method on a background task. And then in the test you wanna say like, I want to. Confirm that this eventually, you know, the method being run in the background that it eventually does its work. And that's precisely what this is doing. like in your example, it's like it keeps calling raised dolphins until the account gets up to one, Is that correct? So what happens is that raised dolphins is called once but.

As part of Okay. yeah, as part of, and as part of like the internal implementation of it, it might increment the number of dolphins on it. And so then you're just checking like that is that eventually there is at least one dolphin there? I assume it would take some sort of time interval or like duration. 'cause yeah. Yeah. really interesting, yeah. That's been like a thing of, that's the, time interval thing of like when to cut this off has been the most eye-opening thing about this

right? Swift Testing submits every single one of your tests to be run at once, and it submits it all to the Swift concurrency shared thread pool. So on your system, you might have that shared thread pool, might have like 10 threads available, but if you have like 300 or so tests, you know, to run, Then it's gonna run those all and they might all be running concurrently, or at least all spend some amount of time on that, shared thread pool before, a wait causes the next one to run.

And so that prevented an issue with the timeout. Or the desire to have this automatically timeout after like one second. If you have 300, 3000, you know, 10,000 or whatever tests to run on it. And then your test gets run, you then like await to as part of this. There's no guarantee. And in fact, the more tests you have, the more likely, and especially in like resource constrained environments like ci there's no guarantee that you'll actually get that callback within that, like one second timeout.

So on like extremely large on extremely large test suites, this will. Polling confirmations as I originally implemented it, I have fixed that now. Will be increasingly flaky and unreliable. Okay. That makes sense. Yeah, disappointing. Yeah. before we close out, a lot of people are curious about like CUI tests. They're still very much tied to the old Objective C APIs and things like that. Where, what do you see as like the future of that as far as

being more Swift friendly, I mean, it's gotta be like Swift Testing support. There's a lot of, it is a lot of those APIs are very extremely typed. So ideally there'll be some way to make them. Not be as much there is. I don't really see how, I guess there is the, what is it, the dynamic callable or like the dynamic property accessor from a while ago that could be used to at least

make it feel more Swifty, even if it doesn't actually provide any guarantees. that might be it in terms of like making it feel more like idiomatic Swift. But like I think what they're more interested in doing is enabling, like using the automation framework with Swift Testing. Okay. Because one of the things that they did, I don't remember whether it was 16 three or 16 four, but they pulled out or Xcode 16, three or four they pulled out the, audit. They made a new XC automation framework, which is separate from XC test, which

is another one of those reading between the lines of like, they're clearly going in the direction of enabling automation tests with Swift Testing. What is the automation framework exactly? It's all of the XCUI test APIs. So it's like the XCUI element, so you can get an element by the, accessibility identifier. it's like the XCUI application and interacting with it through there. Is that used outside of testing for like Mac OS apps.

No, it's not. It's entirely meant for use with automation tests. Because I know like there's apps that I'll always ask for, like accessibility, access. That is a different, that is also it's a different automation library from Apple. Great. so that is much more of like just being able to interact with the system through the. Automation A PII believe. I wouldn't be surprised if XE automation is fundamentally using like those same APIs under the hood,

Under the hood. but I don't have any way to verify that. Anything else you wanted to talk about, especially with dub or testing before we close out? Yeah, The other stuff I'm really excited for all of the, concurrency improvements with Swift six two. I've been diving into those and just been really excited for that. The defaulting to running everything or constraining everything to the main actor. As opposed to non isolation is something that I'm excited to see. Though it is right now very buggy.

But you know, it's also just happened. It's early in the beta season. We'll see how it turns out. Well Rachel, thank you so much for coming on the show. it's great to have you on and talk about this stuff. Where can people find you online? yeah, I am. my blog is at rachel brindle.com. I am on, Mastodon at unata@hackderm.io. And I. Instagram, though, that's mostly flying stuff now. at Rachel Brindle.

Yeah. Thank you again. we'll put links to the proposal, your blog post, all that stuff in the show notes below. People can find me on Mastodon at Leo G. Dion at c Im I'm on Blue Sky as well at Leo g Dion. Take a look at me there. Links are in the show notes. If you really enjoyed this episode, please like and subscribe. Early access is to those who joined Patreon, so thank you for all the Patreon supporters. It's super helpful. Thank you so much and I look forward to talking to you in the next episode.

Bye everybody.

Open the transcript file

Hosts and Guests

Hosts

  • Leo Dion Host Feed-stated

Guests

  • Rachel Brindle Guest Feed-stated

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Not Accessible by Default with Rob Whitaker

    Nov 16, 20261 hr 10 minAudio

    Rob Whitaker joins Leo to talk about why accessibility doesn't come for free, not even with SwiftUI. Rob, an iOS engineer specializing in accessibility, author of Developing Inclusive Mobile Apps, and a member of the...

    Credit the people on this episode

  2. You’re Better Than AI with Daniel Steinberg

    Nov 9, 202651 minAudio

    Daniel Steinberg returns to chat about Apple's Foundation Models framework and what changed in OS 27. We get into using AI only where it's appropriate, experimenting with #Playground, falling back gracefully when the...

    Credit the people on this episode

  3. Do the Wrong Thing Much Faster with Graham Lee

    Nov 2, 20261 hr 12 minAudio

    Graham Lee returns to chat about what it takes to become a software centaur: using AI to get better at the craft, not just faster. We get into why the job was never about typing code, meat proxies and Jevons paradox,...

    Credit the people on this episode

  4. Then 20 Years Later with Georg Tuparev

    Oct 26, 202643 minAudio

    Georg Tuparev stops by to chat about software that outlives its plans: projects scoped for a few months that are still in development 20 years later. We get into running a small team of physicists and astronomers, why...

    Credit the people on this episode

  5. We Don’t Have Any Futures or Promises Anymore with Tim Condon

    Oct 16, 202647 minAudio

    Tim Condon stops by to chat about Vapor 5, the biggest Vapor release in six and a half years. We get into dropping futures and promises for Swift concurrency all the way down the stack, Span and strict memory safety,...

    Credit the people on this episode