Switch to codecov by dmoody256 · Pull Request #3275 · SCons/scons

dmoody256 · 2019-01-19T03:04:56Z

This PR switches to codecov which allows for coverage reports submitted from both Appveyor and Travis so we can get a complete coverage including windows code and tests.

Fixes some minor issues with the files omitted for coverage calculation
disables coveralls and switches to using codecov for coverage reporting
adds py 2.7 and 3.6 on the Visual Studio 2015 image, and maintains py 2.7, 3.5, 3.6, 3.7 on the 2017 image
adds additional coverage for py 3.6 on Travis, and compresses coverage builds into 4 jobs each for speed
calculates coverage even if a single test fails, as this can still be useful information
adds the codecov.yml for source level coverage configuration.
Adds appveyor Ubuntu configuration as an optional environment, but disabled for now

Codecov example report:
https://codecov.io/gh/SCons/scons

bdbaddog · 2019-01-20T04:15:43Z

Why do linux on both appveyor and travis?

dmoody256 · 2019-01-20T04:25:52Z

Why do linux on both appveyor and travis?

I questioned this too, but I guess because redundancy, different environment and versions of 3rd party software.

Alternatively, removing it would speed up the appveyor build time, they only alot one concurrent job for open source projects.

It's almost optional, because to remove it requires only simply commenting out the image at the top of the appveyor yaml.

bdbaddog · 2019-01-20T04:49:32Z

I'd say leave the logic in for appveyor, but comment it out for now? Sounds like this would hold up any other PR's builds until extra runtime on appveyor which is already a the slowest part of the build..

…pveyor

dmoody256 · 2019-01-20T16:32:47Z

I'd say leave the logic in for appveyor, but comment it out for now

ok done, also before merging you will need to go to codecov and click the sign up button to connect the SCons account with github:
https://codecov.io/
then connect the scons repository. Codecov won't need any repo tokens it can receive reports from Travis and Appveyor automatically.

dmoody256 · 2019-01-20T20:50:49Z

regarding the failures in the Visual Studio 2015 appveyor image:

the mingw failure is related to the PR Mingw link issue #3270
the java failures are due to the test framework selecting a different javac (1.7) and java (1.6) from the where_java calls. I wrote up an issue here: Appveyor VS 2015 image fails some Java tests #3276

bdbaddog · 2019-01-20T21:48:56Z

+  ### WINDOWS ###
+  # add python and python user-base to path for pip installs
+  - cmd: "C:\\%WINPYTHON%\\python.exe --version"
+  - cmd: for /F "tokens=*" %%g in ('C:\\%WINPYTHON%\\python.exe -m site --user-base') do (set PYUSERBASESITE=%%g)


Why use user-base?

to avoid any permissions issues

Is this a new requirement? We weren't doing that before..

I tested without --user and there was no issues. I believe the permissions issues where on the linux side and I copied the --user from there as a blanket to cover any permissions issues. The windows side doesn't need --user for pip.

bdbaddog · 2019-01-22T03:02:25Z

Check the appveyor run? some java test failed looking for java version 51?

dmoody256 · 2019-01-22T03:51:39Z

Check the appveyor run? some java test failed looking for java version 51?

That's the java issue I mentioned earlier: #3276. I was looking into it and the binaries are all present on that appveyor image and working, so I think there is something going on with the test framework, haven't nailed it down yet though.

bdbaddog · 2019-01-28T04:11:03Z

Is this supposed to only test with py27 and 36, and not 3.5 or 3.7?

dmoody256 · 2019-01-28T04:15:17Z

Is this supposed to only test with py27 and 36, and not 3.5 or 3.7?

I removed 3.5 and 3.7 because the time it takes to run the CI with one job. Those are tested on Linux in Travis, so I am not sure how much more value they are adding on windows. With this PR as it stands, Appveyor takes 2 hours, instead of the 4 currently. I can add them back if you think its worth it.

bdbaddog · 2019-01-28T18:21:31Z

Since the PR title is switch to codecov, it's be best if this PR only did that and not change anything else.
Smaller PRs that do one thing are the preferred.
So for this PR please change it back to running the same Python versions as before.

bdbaddog · 2019-01-28T18:24:07Z

looks like win32/mingw.py test is failing on appveyor? can you take a look

dmoody256 · 2019-01-28T18:37:04Z

looks like win32/mingw.py test is failing on appveyor? can you take a look

Thats from the issue in PR: #3270

Also the VS 2015 exec test failed related to license expired, I notified Appveyor:
https://help.appveyor.com/discussions/problems/19717-visual-studio-2015-trial-license-has-expired

dmoody256 · 2019-01-28T18:45:20Z

Since the PR title is switch to codecov, it's be best if this PR only did that and not change anything else.
Smaller PRs that do one thing are the preferred.
So for this PR please change it back to running the same Python versions as before.

OK just to make sure I understand, the CI should test py 2.7, 3.5, 3.6, 3.7 on the Visual Studio 2017 image (the current config) and also test py 2.7, 3.6 on Visual Studio 2015 image?

bdbaddog · 2019-01-28T18:56:45Z

Can you update the summary for this PR to indicate all the included changes?
Likely we should update CHANGES.txt since the CI tasks being run is also(?) being changed?

dmoody256 · 2019-01-28T19:18:46Z

Can you update the summary for this PR to indicate all the included changes?

sure

Likely we should update CHANGES.txt since the CI tasks being run is also(?) being changed?

I thought CHANGES.txt was related to src tree changes, and since this is CI is not documented?

bdbaddog · 2019-01-28T19:20:09Z

In this case since it's a change in the environments we regularly test in, probably worth noting.

bdbaddog · 2019-01-28T19:23:19Z

any chance you can pop on IRC?

dmoody256 · 2019-01-28T19:27:34Z

any chance you can pop on IRC?

I have something I need to do for the next few hours but i'll check on IRC when I get a chance.

…s to original plus 2.7 on 2015, add irc notification for codecov

bdbaddog · 2019-01-29T17:20:53Z

Per IRC discussion only expected tests failed. I'm going to merge this and then we'll address the failures with other PRs.
Thanks for the good work!

dmoody256 added 2 commits January 18, 2019 20:53

switch to codecov and add new coverage for appveyor and travis

5f22d60

swtich to codecov badge

55b1e9a

switch badge to scons repo, fix mispelling, commented out linux on ap…

43e2b06

…pveyor

dmoody256 force-pushed the more_coverage branch from cb6982f to 43e2b06 Compare January 20, 2019 16:28

bdbaddog reviewed Jan 20, 2019

View reviewed changes

dmoody256 force-pushed the more_coverage branch from 9899832 to 9b6881a Compare January 28, 2019 19:17

remove user base install, add no progess for reliablity, set test env…

9d1a177

…s to original plus 2.7 on 2015, add irc notification for codecov

dmoody256 force-pushed the more_coverage branch from 9b6881a to 9d1a177 Compare January 28, 2019 23:10

bdbaddog merged commit 43d4002 into SCons:master Jan 29, 2019

dmoody256 mentioned this pull request Jan 29, 2019

Switch stages around to get correct build failures from Travis CI #3282

Merged

Uh oh!

Conversation

dmoody256 commented Jan 19, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bdbaddog commented Jan 20, 2019

Uh oh!

dmoody256 commented Jan 20, 2019

Uh oh!

bdbaddog commented Jan 20, 2019

Uh oh!

dmoody256 commented Jan 20, 2019

Uh oh!

dmoody256 commented Jan 20, 2019

Uh oh!

bdbaddog Jan 20, 2019

Choose a reason for hiding this comment

Uh oh!

dmoody256 Jan 20, 2019

Choose a reason for hiding this comment

Uh oh!

bdbaddog Jan 21, 2019

Choose a reason for hiding this comment

Uh oh!

dmoody256 Jan 21, 2019

Choose a reason for hiding this comment

Uh oh!

bdbaddog commented Jan 22, 2019

Uh oh!

dmoody256 commented Jan 22, 2019

Uh oh!

bdbaddog commented Jan 28, 2019

Uh oh!

dmoody256 commented Jan 28, 2019

Uh oh!

bdbaddog commented Jan 28, 2019

Uh oh!

bdbaddog commented Jan 28, 2019

Uh oh!

dmoody256 commented Jan 28, 2019

Uh oh!

dmoody256 commented Jan 28, 2019

Uh oh!

bdbaddog commented Jan 28, 2019

Uh oh!

dmoody256 commented Jan 28, 2019

Uh oh!

bdbaddog commented Jan 28, 2019

Uh oh!

bdbaddog commented Jan 28, 2019

Uh oh!

dmoody256 commented Jan 28, 2019

Uh oh!

bdbaddog commented Jan 29, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dmoody256 commented Jan 19, 2019 •

edited

Loading