Thoughtworks Go - dreaded 404 on artifact fetching

I've run into this more than I'd care to admit, so figured I should post it up.

Often when chaining Go pipelines I run into a message like:
19:40:33.412 [go] Could not fetch artifact https://ip.ad.dr.es:8154/go/remoting/files/DevCI-Build/852/ExtractCIcomponents/1/ExtractZIP/components/WebPortal.zip. Pausing 15 seconds to retry. Error was : Unsuccessful response '404' from the server

I've found there are two reasons for this message to come up.

  1. Artifact path specified incorrectly
  2. Go server purging accidentally removed the file
An example bad artifact path is visible in the example above.  If you check the pipeline details, you see: 
  • Pipeline Name: #{fetchPipeline}
  • Stage Name: ExtractCIcomponents
  • Job Name: ExtractZIP
  • Source Directory: components/WebPortal
  • Destination: components



What you SHOULD see is:
  • Pipeline Name: #{fetchPipeline}
  • Stage Name: ExtractCIcomponents
  • Job Name: ExtractZIP
  • Source Directory: components/web/WebPortal
  • Destination: components


A subtle but important difference. The source dir MUST match your build artifact location.  In this example, the build artifact looks like:
Source: components/web/WebPortal
Destination: components/web

For the latter example, it's pretty obvious wha happen.
Check here on your Go server:
/var/lib/go-server/artifacts/pipelines/DevelopmentCI-Build/744/ExtractCIcomponents/1/ExtractZip/components/web/

If you no see files n junk...it's been purged.  But there is a way out!!

To regen the files, simply re-run the build from that original pipeline run - the files will re-appear, and any call to 'latest' for that pipeline run will be re-populated.  (although the one time this came up, the code piece built, but the tests piece did not)  It was enough to get us past the production deployment - we could then say 'hay hay hay give Go more disk!'.

Comments

Popular posts from this blog

Learning through failure - a keyboard creation journey

Canary deployments of IIS using Octopus & AWS

Learning Opportunities - Watching/listening list