MSIX and MSIX app attach High-Level Performance Testing

Introduction

This article provides insight into some testing I have completed on MSIX and MSIX app attach. The testing focused on three core areas:

Understanding more about performance and the impact of using the package support framework (PSF).
Taking a deeper look into the performance Metrics of both MSIX and MSIX app attach.
Identifying the performance differences between MSIX images and any specific use cases between the different types.

Do you need to use the Package Support Framework for all MSIX’s

In my opinion, there has been a lot of hype regarding the need to use the Package Support Framework for MSIX and MSIX app attach. In some cases where applications do not function correctly in an MSIX container, you would need to use the Package Support Framework (PSF) to create a fixup. This does not mean you have to use this for all applications. It is suggested that PSF is only needed for applications with complications (problematic apps). I also note that the failure rate is much higher using the Microsoft packaging tool; however, third-party vendors show much higher success rates, with and without PSF.

MSIX partners – MSIX | Microsoft Docs

For those who don’t know much about the Package Support Framework. This is a group of files including a Launcher, fixup dlls, and a Config.JSON file.

Package Support Framework DLL Injection — This image shows a logical representation of the package support framework in use with a application.

The application.exe is called via the launcher.exe, which reads the config.json with the configurations you have specified. You can test the workings of PSF without having to package into an MSIX each time.

Testing Setup:

For this testing, I used a Virtual Machine with 2 vCPU’s, 16GB of Ram and SSD for storage. I packaged a common application which many use on a day to day basis (DM me to find out the app). I created two types of MSIX package: a standard MSIX and the second package that includes the package support framework. To add the PSF components, I unpacked the MSIX package, added the PSF components to the root folder, and then edited the configuration file (.JSON) and updated the package manifest file. Once complete, I repacked the MSIX ready for testing.

It is fairly straight forward to write the PSF config file, and Microsoft provides the PSF monitoring tool for testing and any error checking.

microsoft/MSIX-PackageSupportFramework: The Package Support Framework (PSF) is a kit for applying compatibility fixes to packaged desktop applications. (github.com)

If you need to run PowerShell scripts in your PSF’ed MSIX, you will need to ensure that you have set the execution policy to remote signed. In some organisations, this may be challenging. One way around this would be to code sign each PowerShell script.

Run scripts with the Package Support Framework – MSIX | Microsoft Docs

Testing Scenarios

Testing comprised of two areas, one being application installation and (staging and registering). The second was the actual application process launching. I ran each app test 10 times. I decided to stop at 10 per test as the output was similar after the 5th test.

Install / Staging and Registering:

Installation of a Native application
Install an MSIX
Staging and Registering of MSIX app attach images (all variants)

Application Process Launching:

The following list (type) was tested.

Native app
MSIX
MSIX – (PSF)
MSIX image – VHD
MSIX image – VHDX
MSIX image – CIM
MSIX image – VHD (PSF)
MSIX image – VHDX (PSF)
MSIX Image – CIM (PSF)

Calculating Time:

To calculate the installation or (stage & register) time, I called the process with a start and finish timer, providing me with a start, finish, and time output difference.

When looking at recording the application launching times. I had to write a fancy PowerShell script to ensure I could call the actual MSIX application. When running the script, it grabs the start time, and then I capture the actual start process time within Process Explorer. I then use the difference in time to confirm the process launch time.

Results:

Installation / (Stage & Register)

Native: So, to start testing, I captured the time it took to install a native application. This was approximately 59 seconds.

MSIX: When installing the MSIX for all users, I noted it took 2.4 seconds to install.

MSIX app attach: MSIX averaged around half a second to stage and register for one application.

As you can see graph above, MSIX is much quicker at attaching than installing the MSIX nativity to the host.

Application Process Launch Test Results

These results are interesting. You will note the Package Support Framework MSIX does have a performance impact compared to a standard MSIX package. I also note that MSIX app attach does not provide any improvement to performance for a single user. When using CIMFS, you will note the benefits of delivering MSIX app attach to many users. One other interesting result is that CIMFS did come out on top.

Other Observations

CIMFS uses less CPU and RAM than the other MISX image types. In large production environments, it’s advised that you use CIMFS for MSIX app attach. You will also note from the graph above, CIMFS offers slightly improved Application launch times.
When using WVD, testing has shown that AMD CPU’s provides better application launch times. I wrote a previous article detailing benchmark differences: Sizing and Performance benchmark testing – Windows Virtual Desktop – Ryan Mangan’s IT Blog. It’s worth giving this a try.
I found no significant difference between using MSIX and MSIX app attach in terms of application process launching. The key differences are the speed of delivery to the host.
There is no difference between PSF and non-PFS MSIX packages regarding staging and registering for MSIX app attach.
The first time you launch the MSIX, it is much slower than all subsequent tests. This is because the application is completing its first setup. This can be user data and others.
Please note storage performance and configuration will impact performance stats. If you do see poor performance, check your storage and host specs.
Avoid blanket PSF fix-ups as this will only impact application performance. It is better to fix a specific issue than apply a blanket fix.

Why MSIX app attach:

No agents – Uses the native OS API. The results show that MSIX presents the applications to the host very quickly. (This is game-changing)
Low impact on performance when staging and registering apps. I saw next to no impact when carrying out these tests.
As previously mentioned, MSIX app attach can be used on-premises, WVD and other cloud using Windows 10 build 2004 onwards.
Simple update process, you can roll out updates with users in session.
Isolate users from apps without having to separate images. Reduce image management and control application delivery better.
Store all your applications once. A Central repository, storing everything once. Mitigate application and application version sprawl.

Summary

When using the package support framework, there is a performance impact as you are effectively running a application to run a application.

MSIX app attach provides a much faster stage and register than a native MSIX can install. CIMFS does appear to be very promising and came out on top for both tests Staging/Registering and Application launch.

I found no significant difference in performance in this testing when using VHD, VHDX and CIMFS, MSIX image types. I believe larger pooled application testing is required to realise the benefits and considerations for each.