# AMP Release 4.8.0 * [Introduction](#introduction) * [Alert of Future Changes](#alert-of-future-changes) * [New Features](#new-features) * [Bug Fixes](#bug-fixes) * [Other Changes](#other-changes) * [Known Issues](#known-issues) * [Backwards Compatibility](#backwards-compatibility) * [Upgrade Instructions](#upgrade-instructions) ## Introduction Thank you to our community who have contributed a lot of improvements and feedback! Thanks also go to Brooklyn's commercial users who have funded this development and made some major contributions. For more information, please visit https://cloudsoft.io/amp, https://docs.cloudsoft.io and http://brooklyn.apache.org. This is a major upgrade from AMP version 4.7.0. ## Alert of Future Changes * AMP Cluster is still in beta and grows at a fast pace. A huge number of changes have been added since AMP 4.7.0 to improve the stability. GA for AMP cluster is expected to release at the same time as AMP 5.0.0 will will include enhancements on vertical scaling. * AMP 4.8.0 is the last release of the 4.x series. The next scheduled released will be AMP 5.0.0 and will include exciting new features such as: * GA of AMP Cluster (see previous paragraph) * UI branding and customization - providing tools for customers to brand, customize and extend AMP UI * Policy capabilities: * Failover and notifiers * More operational details on policies and enrichers, including policy-based tasks more visible in `App Inspector` * Catalog bundle management - improve productivity by making bundle the primary way of adding catalog items * Packaging of bundled products ## New Features ### AMP * Karaf CLI has 2 news commands available: * `brooklyn:application-list`: Display a list of deployed applications and their state * `brooklyn:entity-info `: Display detailed information about a single entity ### AMP UI * The app inspector has an improved web notification UI. A new button is now available for users to enable/disable web notification. The browser permission is now deferred when the user actively clicks on the enable button. If the web notifications have been blocked, the button displays this status with a tooltip to explain what is going on and how to enable them again. * Add a confirmation modal when `unmanage` or `expunge` applications. * Add kilt diagram expansion feature. The kilt diagram can now be expanded full size in a modal with a button click. ### AMP Cluster - Beta * Enable configuration of persistence on AMP Workers. It also splits `amp-worker-node` from `amp-worker-stable`. * Add support for catalog zip-upload for the `cluster-api`. * Add support for retry on next worker-amp if a deployment fails. ### Apache Brooklyn Server * Add new REST API endpoint `PUT /applications/{appid}` for deploying an app with a pre-defined app-id. This feature is marked as beta. * Add freemarker DSL. Provides `$brooklyn:template(...)` evaluated against the context entity as a freemarker template. The main advantage of the new DSL is that it's now possible to do simple expressions in YAML - both logical or algebraic. * Add support for simple `min` and `max` aggregators. Also extracts utility functions into `MathAggregatorFunctions` (marking them as `@Beta`), deprecating the nested classes and static methods in `Enrichers`. * Add HttpFeed.preemptiveBasicAuth. For a description of pre-emptive basic auth, see https://hc.apache.org/httpcomponents-client-ga/tutorial/html/authentication.html (section 4.6. Preemptive authentication). As is described in that link, pre-emptive basic auth should be used with caution. However, if you really do need to use basic auth then some servers don't send back the right challenge-response in the headers of the 401 response. For example, this seems to be the case with CouchDB. We've seen in some downstream project people working around it by auto-generating the basic-auth header, and configuring the `HttpRequestSensor` accordingly but this is a bad pattern (e.g. we lose the fact that this is a credential, so would never be able to use externalised configuration to ensure the passwords are not logged or written to persisted state). This will allow those users to delete a bunch of lines of ugly yaml configuration from their blueprints. It will also help people knocking together a proof of concept - they can use `preemptiveBaicAuth` while they ask the people responsible for the servers to fix it properly. ### Apache Brooklyn CLI * Add flags for verbose output of HTTP request data. New global flags `--verbose` and `--vverbose` (very verbose) configure the network library to write data about API requests to stderr. The intention is to make it easier to debug both the working of the CLI and unexpected responses from Brooklyn servers. Example 1: $ ./bin/br --verbose deploy ./bp.yaml POST /v1/applications HTTP/1.1 Host: localhost:8081 User-Agent: Go-http-client/1.1 Content-Length: 0 Authorization: ****** Content-Type: application/json Accept-Encoding: gzip HTTP/1.1 201 Created Transfer-Encoding: chunked Cache-Control: no-cache, no-store Content-Type: application/json Date: Thu, 27 Jul 2017 14:12:56 GMT Expires: 0 Location: http://localhost:8081/v1/applications/sr9uw9bdbu Pragma: no-cache Server: Jetty(9.2.13.v20150730) Vary: Accept-Encoding Id: | sr9uw9bdbu Name: | Application (sr9uw9bdbu) Status: | In progress Example 2: $ ./bin/br --vverbose application vhkhp6nbbl entity GET /v1/applications/vhkhp6nbbl/entities HTTP/1.1 Host: localhost:8081 User-Agent: Go-http-client/1.1 Accept: application/json, text/plain Authorization: ****** Accept-Encoding: gzip HTTP/1.1 200 OK Transfer-Encoding: chunked Cache-Control: no-cache, no-store Content-Type: application/json Date: Thu, 27 Jul 2017 14:16:43 GMT Expires: 0 Pragma: no-cache Server: Jetty(9.2.13.v20150730) Vary: Accept-Encoding 503 [{"id":"tqj1a2g883","name":"Tomcat 7 Server","type":"org.apache.brooklyn.entity.webapp.tomcat.TomcatServer","catalogItemId":"org.apache.brooklyn.entity.webapp.tomcat.TomcatServer:0.12.0-SNAPSHOT","links":{"self":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883","parent":"/v1/applications/vhkhp6nbbl/entities/vhkhp6nbbl","application":"/v1/applications/vhkhp6nbbl","children":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/children","config":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/config","sensors":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/sensors","effectors":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/effectors","policies":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/policies","activities":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/activities","locations":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/locations","tags":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/tags","expunge":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/expunge","rename":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/name","spec":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/spec","iconUrl":"/v1/applications/vhkhp6nbbl/entities/tqj1a2g883/icon","catalog":"/v1/catalog/entities/org.apache.brooklyn.entity.webapp.tomcat.TomcatServer/0.12.0-SNAPSHOT"}}] 0 Id | Name | Type tqj1a2g883 | Tomcat 7 Server | org.apache.brooklyn.entity.webapp.tomcat.TomcatServer The Authorization header is redacted. ## Bug Fixes ### AMP * Update default catalog for AMP Cluster by adding missing amp-cluster catalog imports. ### AMP UI * Make the graph in the `Blueprint Composer` responsive. This means the graph won't change size even if the browser zoom is used. It will also preserve the graph position independently. * Use the `iconUrl` field returned by the API to display location icon (if exists) in the `Location Manager`. * Make the deploy link with an hand pointer instead of classic mouse pointer. * Fixed hyperlink URIs in the config and sensor table. * Clear correct sensors (`service.notUp.indicators` and `service.problems`) when resetting entity problems. * Fix links to catalog item and deploy feature when importing BOM in the `Blueprint Importer`. * Fix sensor values containing '@' such that they are turned into mailto links. * Fix addition of custom config keys in `Location Manager`. Now when hitting Enter with a new custom config key, it will be added with an empty value for editing. ### AMP Networking * Fix timeout while waiting for entities to provision ### Apache Brooklyn CLI * Fix handling of global flags in scope definition. Previously the presence of global flags in a command like `br --flag app a ent b sensor c` caused the CLI to run the command for app, not sensor. ### Apache Brooklyn Library * Fix MariaDBNode default mirrorUrl. The existing default mirror does not keep all the old versions. * Convert AnsibleEntity and SaltEntity to extend from `SoftwareProcess`. The current implementation uses `EffectorStartableImpl` instead of `SoftwareProcessImpl`. This has some side effects which might be surprising for users, such as: * the inboundPort (provisioning.properties) are not applied when the SG is created (on supported clouds) * the auto-opening port on SG by creating config keys finishing by .port does not works * the shell.env are also not applied (BROOKLYN-503) * if the location is using a machine name customiser, this will throw a NPE as the CALLER_CONTEXT is not set This is now resolved. ### Apache Brooklyn Server * Remove `org.ow2.asm`, which was already included in Karaf. This was breaking the Karaf distribution. ## Other Changes ### AMP * Update `brooklyn-couchdb-cluster` version to 1.0.3. * Update jClouds labs `azure` provider to 2.0.2.1-20170825.0953. 2.0.2.1-20170825.0953 is a custom release of jClouds labs `azure` to include this fix: https://github.com/jclouds/jclouds-labs/pull/409. This fixes an issue where extra resources are not being cleaned up. Orphaned networks and security groups may still not be cleaned up. * Update jClouds `openstack-nova` provider to 2.0.2.4-20170823.1339. 2.0.2.4-20170823.1339 is a custom release of jClouds `openstack-nova` to include this fix: https://github.com/jclouds/jclouds/pull/1128. This adds validation over the `templateOption.securityGroups` passed by the user. It also accepts `templateOption.securityGroupNames`. * Add custom jClouds `packet` provider, version 2.0.2.2-20170828.1139. 2.0.2.2-20170828.1139 is a custom release of jClouds labs `packet` to include this fix: https://github.com/jclouds/jclouds/pull/1130. This fixes an issue that prevented deployments succeeding, due to a change onto the Packet API. * Add default SSL config for AMP. This uses a default self-signed certification. ### AMP UI * Update build to enable long term caching. Should improve long term caching as all vendors source code is now separated from the app code. * Update auto-update and add log tail features for streams. The logs for ssh commands now auto-update and tail the latest updates. The style has also been changed to look nicer. ### AMP Cluster - Beta * Improve the sequence of actions taken by api server on deploy app. * Reduce bcrypt strength. When strength is high, it can in some conditions max out the CPU massively. For instance, tests have shown that CPU usage is close to 100% while repeatedly checking password hashes with bcrypt (in a tight loop in a single thread). * Improve resilience by replacing a failed `amp-worker`. ### Apache Brooklyn Server * Add a simple timer for brooklyn init and logs duration. * Add cluster.max.size to dynamic cluster, and prevent the cluster from being manually increased beyond that size. * AutoScalerPolicy to resize to limits on expunge. Previously, if a user resized a cluster with an `AutoScalerPolicy` such that the pool size was outside the min and max pool size for the policy, the cluster was not resized to the bounds. * Add `suppressDuplicates` config key for yaml sensors. Used whether to publish the sensor value again, if it is the same as the previous value. * HttpRequestSensor: handling non-json. Previously the `HttpRequestSensor` failed if no `jsonPath: ` is suppiled in the yaml. Added support for setting `jsonPath` to blank, so that it just takes the pure-string value (i.e. no json parsing). * Add `DynamicCluster.START_TIMEOUT`. Similar to `SoftwareProcess`, this allows a dynamic cluster to wait for `service.isUp` before returning from `start()`. It defaults to not waiting (for backwards compatibility). Previously, it would return as soon as all the initial member’s `start()` effectors had finished executing. * Add support to configure `producer` in `UpdatingMap` enricher. * Support DSL in template options. Previously, it was not possible to specify DSL in template options - for instance, you couldn't do: - type: some-type-publishing-subnet-id id: ent1 - type: some-service id: service brooklyn.config: provisioning.properties: templateOptions: subnetId: $brooklyn:entity("ent1").attributeWhenReady("subnet.id") This adds support in `TemplateOptionsOption` for resolving the values supplied for the options. This allows the approach in the blueprint above to be used. ### QA Test Framework * Update the URL to use the new one to download `br` CLI. ### Clocker * Update node setup to the newest version. Previously, Clocker was using node 0.12. This could cause deployments to fail as node 0.12 is EOL. ## Known Issues * NodeJS is currently broken on CentOS, prior 7.4. This is due to one of its dependencies not being in EPEL yet. This issue requires no further actions from you nor Cloudsoft and will be fix as soon as `http-parser` hits EPEL. In the meantime, any deployment using `nodejs` package on CentOS will fail to deploy. For more information: https://bugzilla.redhat.com/show_bug.cgi?id=1481470. * Under particular conditions, jClouds can fail to resolve the underlying OS version for CentOS, while querying for the list of available images. This can cause deployments to fail. For More information: https://github.com/jclouds/jclouds/pull/1041. * There are known issues when using some of the location customization config keys when provisioning Windows machines. * BROOKLYN-529: Windows machine doesn't provision if keyPair is specified. * BROOKLYN-530: InitScript for Windows machine doesn't run when USER_METADATA_STRING or USER_METADATA_MAP are defined. * BROOKLYN-531: When using the DockerContainer entity on a Kubernetes location, the public IP address of the k8s cluster is populated incorrectly. The private IP is used instead. ## Backwards Compatibility * This release only supports Java SE 8. * For backwards compatibility with 3.x, please see the release notes for AMP 4.0.0. * For backwards compatibility with 4.x please see the intermediate release notes ## Upgrade Instructions * See detailed upgrade instructions at https://docs.cloudsoft.io/operations/upgrade.html * Note that from AMP 4.3 Java SE 8 is required * Use of RPM and DEB is now recommended, rather than the tar.gz. * CentOS 7.x is recommended, though CentOS 6 and Ubuntu 16.04 are also tested. * If binding to existing persisted state, an additional command is required to update the existing catalog with the latest AMP versions. Assuming AMP has been installed to /opt/amp (as is done by the RPM and DEB): br catalog add /opt/amp/catalog/catalog.bom