Concourse v5.8.0 Release Notes

Release Date: 2020-01-08 // about 4 years ago
  • ๐Ÿฑ ๐Ÿ”— feature

    The first step (heh) along our road to v10 has been taken!

    @evanchaoli implemented the set_pipeline step described by RFC #31. The RFC is still technically in progress so the step is 'experimental' for now.

    ๐Ÿ— The set_pipeline step allows a build to configure a pipeline within the build's team. This is the first "core" step type added since the concept of "build plans" was introduced, joining get, put, and task. Exciting!

    ๐Ÿ”ง The key goal of the v10 roadmap is to support multi-branch and PR workflows, which require something more dynamic than fly set-pipeline. The theory is that by making pipelines more first-class - allowing them to be configured and automated by Concourse itself - we can support these more dynamic use cases by leveraging existing concepts instead of adding complexity to existing ones.

    As a refresher, here's where this piece fits in our roadmap for multi-branch/PR workflows:

    ๐Ÿ— With RFC #33: archiving pipelines, any pipelines set by a set_pipeline step will be subject to automatic archival once a new build of the same job completes that no longer sets the pipeline. This way pipelines that are removed from the build plan will automatically go away, while preserving their build history.

    ๐Ÿ— With RFC #34: instanced pipelines, pipelines sharing a common template can be configured with a common name, using ((vars)) to identify the instance. For example, you could have many instances of a branches pipeline, with ((branch_name)) as the "instance" var. Building on the previous point, instances which are no longer set by the build will be automatically archived.

    โšก๏ธ With RFC #29: spatial resources, the set_pipeline step can be automated to configure a pipeline instance corresponding to each "space" of a resource - i.e. all branches or pull requests in a repo. This RFC needs a bit of TLC (it hasn't been updated to be prototype-based), but the basic idea is there.

    With all three of these RFCs delivered, we will have complete automation of pipelines for branches and pull requests! For more detail on the whole approach, check out the original v10 blog post.

    Looking further ahead on the roadmap, RFC #32: projects proposes introduce a more explicit GitOps-style approach to configuration automation. In this context the set_pipeline step may feel a lot more natural. Until then, the set_pipeline step can be used as a simpler alternative to the concourse-pipeline resource, with the key difference being that the set_pipeline step doesn't need any auth config.

    ๐Ÿฑ ๐Ÿ”— feature

    • @evanchaoli added support for var_sources in the pipeline config. With this feature, concourse can fetch secrets from multiple independent credential managers per pipeline. While this feature is currently in an experimental state and not yet tested in production, it is the first step to enabling workflows where teams sharing a Concourse instance can independently manage their own credential managers. For the moment, only vault or the dummy credential manager can be used to back a var_source (the other credential manager types do not work). #4600, #4777

    ๐Ÿฑ ๐Ÿ”— feature

    • ๐Ÿ”ง @evanchaoli added the ability to tune the mapping between API actions and roles via the --config-rbac flag. While you can't yet create your own roles, you can customize the built-in ones by promoting and demoting the roles to which certain API actions are assigned. #4657

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿฑ ๐Ÿ”— feature

    • ๐Ÿ“Œ The pin menu on the pipeline page now matches the sidebar, and the dropdown toggles on clicking the pin icon. #4688

    ๐Ÿฑ ๐Ÿ”— feature

    • Prometheus and NewRelic can receive Lidar check-finished event now. #4556

    ๐Ÿฑ ๐Ÿ”— feature

    • ๐Ÿ”ง Make Garden client HTTP timeout configurable. #4707

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿฑ ๐Ÿ”— feature

    • ๐Ÿ— @hfinucane added a --url flag to fly watch, so now you can just copy the URL of a build from your browser and paste it in your terminal to keep watching the build. #4323

    ๐Ÿฑ ๐Ÿ”— feature

    • Concourse team roles can now be assigned to different CF space roles independently. For example, you can now create role mappings like "auditors in my CF space should be viewers in my Concourse team", whereas before you could only assign Concourse roles to CF developers. #4712, #4729

    ๐Ÿฑ ๐Ÿ”— feature

    • ๐Ÿš€ Concourse now emits some useful metrics when lidar is enabled: the size of the check queue, the number of checks queued per atc each tick, number of checks GCed at a time, checks started and checks finished. #4692

    ๐Ÿฑ ๐Ÿ”— feature

    • ๐Ÿ— The build page now shows text labels for different step types, like get:, task: and set_pipeline:, instead of the icons from previous versions. Hopefully this is more accessible and easier to interpret! #4942

    ๐Ÿฑ ๐Ÿ”— feature, stub

    • The Concourse team is in the early stages of implementing a new backend for our container runtime based on containerd, which is more featureful than the guardian we have relied on until now. We have not yet implemented all of the methods required by Garden, so the existing work (which can be enabled by passing the --use-containerd flag to concourse worker) is in a non-functional state. This work is tracked in this project. #4779, #4778, #4752, #4853, #4784

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ›  @kcmannem finally fixed the jagged edges on the progress bar indicators used by the dashboard. #4865

    ๐Ÿฑ ๐Ÿ”— fix

    ๐Ÿ— @evanchaoli fixed a weird behavior with secret redaction wherein a secret containing e.g. { on its own line (i.e. formatted JSON) would result in { being replaced with ((redacted)) in build logs. Single-character lines will instead be skipped. #4749

    As an aside, anyone with a truly single-character credential may want to add another character or two.

    ๐Ÿฑ ๐Ÿ”— fix

    • 0๏ธโƒฃ @vito bumped the autocert dependency so that Let's Encrypt will default to the ACME v2 API. #4804

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿš€ Bumped the registry-image resource to v0.8.2, which should resolve DIGEST_INVALID errors (among others) introduced by faulty retry logic. Additionally, the resource will now retry on 429 Too Many Requests errors from the registry, with exponential back-off up to 1 hour.

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ›  @evanchaoli fixed a race condition resulting in a crash with LIDAR enabled. #4808

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ— @evanchaoli fixed a regression introduced with the secret redaction work which resulted in build logs being buffered. #4817

    ๐Ÿฑ ๐Ÿ”— fix

    • Fixed the problem of when fail_fast for in_parallel is true, a failing step causes the in_parallel to fall into on_error. #4746

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿšš @witjem removed superfluous mentions of register-worker from TSA. #4816

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ”ง @evanchaoli changed the behaviour of fly set-team so that when a role has no groups or users configured, it no longer raises an error. #4858

    ๐Ÿฑ ๐Ÿ”— fix

    • @witjem improved the error that fly reports when your .flyrc has invalid YAML. #4758

    ๐Ÿฑ ๐Ÿ”— fix

    • @xtremerui changed the concourse CLI to output help text on stdout when the -h or --help flag is passed. This makes it easier to use other tools like grep to find relevant parts of the usage text. #4745

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ“œ Concourse used to check the existence of legacy migration table by accessing information_schema and parsed out the error message does not exist in English; @xtremerui changed it by using to_regclass in postgres 9.4+, which resolved the issue for users with non-English (i.e. German) system language setup failed to migrate database. #4701

    ๐Ÿฑ ๐Ÿ”— fix

    โœ… @vito bumped the default value for the Let's Encrypt ACME URL to point to their v2 API instead of v1. This should have been in v5.7.2, but we had no automated testing for Let's Encrypt integration so there wasn't really a mental cue to check for this sort of thing.

    โœ… We're adding Let's Encrypt to our smoke tests now to catch API deprecations more quickly, and a unit test has been added to ensure that the default value for the ACME URL flag matches the default value for the client. #4869

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ›  @pivotal-bin-ju fixed x509 issue when the super admin login without CACert after the first sucessful login. #4587

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ‘ท @kirillbilchenko fixed a bug where the concourse_workers_registered metric would never go below 1, even when workers were pruned. #4895

    ๐Ÿฑ ๐Ÿ”— enhancement

    • ๐Ÿ— @matthewpereira enlarged the build prep list font to match the other build log output styling. #4826

    ๐Ÿฑ ๐Ÿ”— fix

    • ๐Ÿ›  @cirocosta fixed a bug where an error that's not specific could lead to null pointer exception during the container creation phase. #4932