Skip to content
This repository has been archived by the owner on Jan 17, 2023. It is now read-only.

Updated Rakefile to run multiple OS versions of unit tests #2716

Merged
merged 1 commit into from May 14, 2015
Merged

Updated Rakefile to run multiple OS versions of unit tests #2716

merged 1 commit into from May 14, 2015

Conversation

kcharwood
Copy link
Contributor

#2702 helped me discover our unit tests aren't running on all of the available simulators installed on Travis.

This PR attempts to rectify that by using the -destination parameter for Xcode build. You can get the list of available simulators on a machine by using xcrun simctl list, and I have some regex to pull out a simulator for each OS version.

This may still occasionally cause Travis to fail because of the issues with the reachability tests that are fixed in #2702.

cc @cnoon @mattt @kylef

@kcharwood kcharwood force-pushed the unit_test_rake_file_updates branch from 1d1af30 to 71989df Compare May 12, 2015 19:41
@kcharwood kcharwood changed the title Updated Rakefile to run multiple versions of unit tests Updated Rakefile to run multiple OS versions of unit tests May 12, 2015
@kcharwood
Copy link
Contributor Author

Looks like this one will need a bit more work as well.

Travis Error Log:

xcodebuild -workspace AFNetworking.xcworkspace -scheme 'iOS Tests' -sdk 'iphonesimulator' -destination 'platform=iOS Simulator,OS=7.0,name=iPhone 4s' -destination 'platform=iOS Simulator,OS=7.1,name=iPhone 4s' -destination 'platform=iOS Simulator,OS=8.1,name=iPhone 4s' -configuration Release clean test | xcpretty -c ; exit ${PIPESTATUS[0]}
xcodebuild: error: Unable to find a destination matching the provided destination specifier:
        { platform:iOS Simulator, OS:7.0, name:iPhone 4s }
    The requested device could not be found because no available devices matched the request.
    Available destinations for the "iOS Tests" scheme:
        { platform:iOS Simulator, id:FFDC0F02-6F2D-4261-A188-36C04B845F57, OS:7.0.3, name:iPad 2 }
        { platform:iOS Simulator, id:27281F5C-9697-4009-9932-722486AFBBD4, OS:7.1, name:iPad 2 }
        { platform:iOS Simulator, id:32D0747C-DAF9-4EA6-BC66-28EAE7CA357A, OS:8.1, name:iPad 2 }
        { platform:iOS Simulator, id:0A5D20B3-5D34-46F5-99B3-EB4AEA113421, OS:7.0.3, name:iPad Air }
        { platform:iOS Simulator, id:88AAFE0A-8784-4CDF-997A-4387E38CF85A, OS:7.1, name:iPad Air }
        { platform:iOS Simulator, id:33309D21-9B76-41F2-B16E-8F9B4DB269AE, OS:8.1, name:iPad Air }
        { platform:iOS Simulator, id:62A308F2-3272-4EEA-9CD1-90C2335FD3E8, OS:7.0.3, name:iPad Retina }
        { platform:iOS Simulator, id:215C68E1-4BE6-4169-B30D-1847CF744474, OS:7.1, name:iPad Retina }
        { platform:iOS Simulator, id:78CCB429-8012-4AB9-BB00-48C945BA1722, OS:8.1, name:iPad Retina }
        { platform:iOS Simulator, id:16E214BA-9A36-489E-81C5-9CCB9604FF8B, OS:7.0.3, name:iPhone 4s }
        { platform:iOS Simulator, id:DF1BBEDE-90C5-41D9-A11B-8DA659322101, OS:7.1, name:iPhone 4s }
        { platform:iOS Simulator, id:37BC3125-3F35-4B1B-9195-BC5105B115AF, OS:8.1, name:iPhone 4s }
        { platform:iOS Simulator, id:B3581D0A-B151-4F5E-9884-FC5A78CDD908, OS:7.0.3, name:iPhone 5 }
        { platform:iOS Simulator, id:A3312623-A36D-42FD-BB07-DD817F199572, OS:7.1, name:iPhone 5 }
        { platform:iOS Simulator, id:2DDCAE77-D0F8-4FB1-B9E0-3AFBECA68164, OS:8.1, name:iPhone 5 }
        { platform:iOS Simulator, id:CC4EE950-AC3A-40B9-A360-AD393B8D3D08, OS:7.0.3, name:iPhone 5s }
        { platform:iOS Simulator, id:9262546F-329F-464B-9706-CD956CE4EC88, OS:7.1, name:iPhone 5s }
        { platform:iOS Simulator, id:604674C2-08C7-4F3E-9910-BB7731045B9A, OS:8.1, name:iPhone 5s }
        { platform:iOS Simulator, id:0569A245-BFFC-4F9D-96BD-BF4D5E28F738, OS:8.1, name:iPhone 6 Plus }
        { platform:iOS Simulator, id:28727EC6-4665-4FC2-B77C-C2C456206044, OS:8.1, name:iPhone 6 }
        { platform:iOS Simulator, id:115F4F5D-C861-4342-A274-D8438126F158, OS:8.1, name:Resizable iPad }
        { platform:iOS Simulator, id:90F9F633-C4A2-499C-82C0-0E2CD82A1829, OS:8.1, name:Resizable iPhone }
! iOS unit tests failed
The command "rake test" exited with 70.

@kcharwood
Copy link
Contributor Author

Here are the available simulators on Travis:

== Device Types ==
iPhone 4s (com.apple.CoreSimulator.SimDeviceType.iPhone-4s)
iPhone 5 (com.apple.CoreSimulator.SimDeviceType.iPhone-5)
iPhone 5s (com.apple.CoreSimulator.SimDeviceType.iPhone-5s)
iPhone 6 Plus (com.apple.CoreSimulator.SimDeviceType.iPhone-6-Plus)
iPhone 6 (com.apple.CoreSimulator.SimDeviceType.iPhone-6)
iPad 2 (com.apple.CoreSimulator.SimDeviceType.iPad-2)
iPad Retina (com.apple.CoreSimulator.SimDeviceType.iPad-Retina)
iPad Air (com.apple.CoreSimulator.SimDeviceType.iPad-Air)
Resizable iPhone (com.apple.CoreSimulator.SimDeviceType.Resizable-iPhone)
Resizable iPad (com.apple.CoreSimulator.SimDeviceType.Resizable-iPad)
== Runtimes ==
iOS 7.0 (7.0.3 - 11B507) (com.apple.CoreSimulator.SimRuntime.iOS-7-0)
iOS 7.1 (7.1 - 11D167) (com.apple.CoreSimulator.SimRuntime.iOS-7-1)
iOS 8.1 (8.1 - 12B411) (com.apple.CoreSimulator.SimRuntime.iOS-8-1)
== Devices ==
-- iOS 7.0 --
    iPhone 4s (16E214BA-9A36-489E-81C5-9CCB9604FF8B) (Shutdown)
    iPhone 5 (B3581D0A-B151-4F5E-9884-FC5A78CDD908) (Shutdown)
    iPhone 5s (CC4EE950-AC3A-40B9-A360-AD393B8D3D08) (Shutdown)
    iPad 2 (FFDC0F02-6F2D-4261-A188-36C04B845F57) (Shutdown)
    iPad Retina (62A308F2-3272-4EEA-9CD1-90C2335FD3E8) (Shutdown)
    iPad Air (0A5D20B3-5D34-46F5-99B3-EB4AEA113421) (Shutdown)
-- iOS 7.1 --
    iPhone 4s (DF1BBEDE-90C5-41D9-A11B-8DA659322101) (Shutdown)
    iPhone 5 (A3312623-A36D-42FD-BB07-DD817F199572) (Shutdown)
    iPhone 5s (9262546F-329F-464B-9706-CD956CE4EC88) (Shutdown)
    iPad 2 (27281F5C-9697-4009-9932-722486AFBBD4) (Shutdown)
    iPad Retina (215C68E1-4BE6-4169-B30D-1847CF744474) (Shutdown)
    iPad Air (88AAFE0A-8784-4CDF-997A-4387E38CF85A) (Shutdown)
-- iOS 8.1 --
    iPhone 4s (37BC3125-3F35-4B1B-9195-BC5105B115AF) (Shutdown)
    iPhone 5 (2DDCAE77-D0F8-4FB1-B9E0-3AFBECA68164) (Shutdown)
    iPhone 5s (604674C2-08C7-4F3E-9910-BB7731045B9A) (Shutdown)
    iPhone 6 Plus (0569A245-BFFC-4F9D-96BD-BF4D5E28F738) (Shutdown)
    iPhone 6 (28727EC6-4665-4FC2-B77C-C2C456206044) (Shutdown)
    iPad 2 (32D0747C-DAF9-4EA6-BC66-28EAE7CA357A) (Shutdown)
    iPad Retina (78CCB429-8012-4AB9-BB00-48C945BA1722) (Shutdown)
    iPad Air (33309D21-9B76-41F2-B16E-8F9B4DB269AE) (Shutdown)
    Resizable iPhone (90F9F633-C4A2-499C-82C0-0E2CD82A1829) (Shutdown)
    Resizable iPad (115F4F5D-C861-4342-A274-D8438126F158) (Shutdown)

As you can see, the 7.0 devices are really 7.0.3 devices, but Apple doesn't list them that way. I have updated the script to associate the runtime version with the OS version from the Runtimes section, and use that value instead.

@kcharwood kcharwood force-pushed the unit_test_rake_file_updates branch from 20e6ac6 to 0184a84 Compare May 12, 2015 21:01
@kcharwood kcharwood added this to the 2.5.4 milestone May 12, 2015
@kcharwood kcharwood self-assigned this May 12, 2015
@kcharwood kcharwood force-pushed the unit_test_rake_file_updates branch from 0184a84 to 80de78b Compare May 13, 2015 14:55
@cnoon cnoon mentioned this pull request May 14, 2015
Merged
@cnoon
Copy link
Member

cnoon commented May 14, 2015

I noticed the following issue being thrown in Travis:

linker error

Rather than explain how to fix it, I just decided to actually fix it in #2719.

@cnoon
Copy link
Member

cnoon commented May 14, 2015

These changes look good to me @kcharwood. My rubyfu isn't stellar, but I'd say this looks like it's getting the job done.

kcharwood added a commit that referenced this pull request May 14, 2015
@kcharwood
Merge pull request #2716 from AFNetworking/unit_test_rake_file_updates
7839c70 
Updated Rakefile to run multiple OS versions of unit tests
@kcharwood kcharwood merged commit 7839c70 into master May 14, 2015
@kcharwood kcharwood deleted the unit_test_rake_file_updates branch May 14, 2015 15:14
@kylef
Copy link
Contributor

kylef commented May 14, 2015

Looks good, perhaps we can look at using https://github.com/layerhq/xctasks to make this simpler in the future.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees

@kcharwood kcharwood

Labels
needs feedback tests
Projects
None yet
Milestone
2.5.4
3 participants
@kcharwood @cnoon @kylef