# AMP Release 4.5.0 * Introduction * Alert of Future Changes * New Features * Bug Fixes * Other Changes * Known Issues * Backwards Compatibility * Upgrade Instructions ## Introduction Thank you to our community who have contributed a lot of improvements and feedback! Thanks also go to Brooklyn's commercial users who have funded this development and made some major contributions. For more information, please visit http://www.cloudsoft.io/amp, http://docs.cloudsoft.io and http://brooklyn.apache.org This is a major upgrade from AMP version 4.4.0. ## Alert of Future Changes * Support for the Azure Classic (`jclouds:azurecompute`) location will be stopped in AMP 4.6 in favour of the newer Azure ARM (`jclouds:azurecompute-arm`). * Support for the Alien4Cloud (https://alien4cloud.github.io/) UI integration will be stopped in AMP 4.6 * Use of internal Groovy in AMP is depreciated and will be removed in future versions. ## New Features * AMP has been updated to use the latest JClouds release, 2.0.1. This adds improvements to the Azure ARM location and compatability with beta providers such as Vagrant (https://github.com/jclouds/jclouds-labs/tree/master/vagrant). * AMP Persistence and Vault keys can now be encrypted with environment credentials. * Additional options to enable setting the random number generator seed for identifiers. Define `brooklyn.experimental.feature.random_seed` with the value of the seed for the random number generator. This means that when locations and entities are created in the same order after startup, they will have the same predictable identifiers, which is useful for testing or demonstrations. * A new Sequencer entity (SequenceEntity) and a group (SequenceGroup) that manages sequential sensor values to entities across an application. * The quick-launch modal has been added into the catalog for quick deployment of apps and entities from the catalog * Clean-up of quick launch items to leave core blueprints. * Elastic search, Logstash and Kibana blueprint added to the quick-launch items. * Jenkins blueprint added to the quick-launch items. * Obscuring vault credentials in `etc/brooklyn.cfg` https://docs.cloudsoft.io/operations/configuration/obscuring-external-supplier.html * Obscuring persisted state https://docs.cloudsoft.io/operations/persistence/obscuring-persisted-state.html * Ability to deploy Windows on deprecated Azure Classic. A templateOptions parameter `winrmUseHttps: true` enables the use of winrm over https in Azure Classic as in AMP 3.5.1. ## Bug Fixes * Update random passwords to use Identifiers.makeRandomPassword rather than Strings.makeRandomId which uses an insecure random. * Fix node.js requirements as newer versions of Ubuntu do not have npm installed by default, this adds it to the install list. * Update executeScript effector for MariaDB and PostreSQL. * Fix address publish bug. When a server provisions but JClouds fails to connect to it, the server is killed and re-provisioned. When this happens the private and public address of the old server remain in the `host.address` sensors because of a null check in `DefaultConnectivityResolver`. The machine now correctly obtains the address of the re-provisioned machine. * Fixed rebind of DSL instances. Previously xstream was injecting the wrong class loader, so we’d get errors like: ``` ClassNotFoundException: org.apache.brooklyn.camp.brooklyn.spi.dsl.methods.DslComponent$DslConfigSupplier ``` * BROOKLYN-453: Fixes for various rebind problems * BROOKLYN-493: Fixes rebind for WinRmCommandSensor * Fixed BlobStoreContextFactoryImpl. It can now have dsl in its config Fixes a bug introduced in this PR https://github.com/apache/brooklyn-server/commit/013b38bf0f146114d3a67423c0dcc3661b50f88c#diff-30cb47bc0f055776b4a246abeddd4ad7L99 Before that change we would ask the JCloudsLocation directly for config. JCloudsLocation uses a ResolvingConfigBag which will resolve dsl (e.g. "$brooklyn:external"). After the above change, we get the ConfigBag from the location first and ask the ConfigBag for the result. When we get the ConfigBag from the location we create a new ConfigBag to return, but we do not create a resolving config bag. This is changed back to asking the location directly for config. * BROOKLYN-476: Externalized config fails for BYON location with windows machines * BROOKLYN-475: fix machine-details parsing * Enable older Java versions for `JavaSoftwareProcess` which are not available by standard on Ubuntu 16.x+. This adds the repo for older versions required by some software. * BROOKLYN-474: rebind fails for Zookeeper policy with private constructor * BROOKLYN-446: deprecate broken ScheduledTask constructors See https://issues.apache.org/jira/browse/BROOKLYN-446 * BROOKLYN-461: fix CollectionMerger’s Ref.equals * Fix Transformer’s DSL resolving for function vals * AMP-1654: Azurecompute Classic and Azurecompute ARM failed to provision windows, this is now fixed * BROOKLYN-482: Config Key default value (in yaml) of type List is mutable * BROOKLYN-444: Effector call works from GUI but fails from CLI ## Other Changes * Deprecate groovy As discussed on the dev@brooklyn mailing list (subject "[PROPOSAL] Remove groovy dependency/support"). Deprecate methods that take explicit groovy types (e.g. `groovy.lang.Closure`, `groovy.time.TimeDuration`, etc). Also updates some javadoc that mentioned `Closure` to say that its usage is deprecated. In some places, it does a `LOG.warn()` about the deprecation when a groovy Closure etc is passed in - this is when we can't just deprecate the method, e.g. for use of `TypeCoercion` on when the method takes `Object` and does an ugly `instanceof`. This does not touch: * The `org.apache.brooklyn.rest.resources.ScriptResource` (for groovy-script execution) * Internal usage of groovy (e.g. `AbstractManagementContext.invokeEffectorMethodLocal()` calling `GroovyJavaMethods.invokeMethodOnMetaClass`) * Remove various REST API end-points deprecated in previous versions of AMP. * yaml-test assertions: fix use of “matches” Previously `matches: ` would not work well for multi-line. For example, if using something like: ``` type: TestSshCommand brooklyn.config: assert.out: - matches: ".*foo.*" ``` and the ssh command outputs: ``` foo1 foo2 ``` then it would not match. You can see this in pure java: `"foo1\nfoo2\n".matches(".*foo.*")` is false. This release changes our "matches" to do two things: first it tries to match the regex against the entire string; but if that does't work then it splits the input string into multiple lines and checks if the regex matches against any single line - if it does, then it considers it a match. * BROOKLYN-460: Brooklyn Camp syntax for adding tags to an entity spec ```yaml services: - type: BasicApplication brooklyn.tags: [tag1, tag2] ``` `brooklyn.tags` documents a list of tag objects which should be assigned to the entity. _Tags can be inspected via REST API `/v1/applications//entities//tags`_ * Fix Docker logo to use transparent background. Docker logo was using incorrect image from official media pack. Changed to the version with a transparent background. * HttpCommandEffector now allows string payload * WinRmMachineLocation obtaining WinRmTool parameters via its config().get - This allows WinrmTool props to be supplied via external configuration providers. - WinRmTool instantiation is made to be the same as in SshMachineLocation#connectSsh * WinRmCommandSensor default executionDir Use default dir obtained on the winrm sesion until Apache Brooklyn has working concept for RUN_DIR on VanillaWindowsProcess * Add status to amp node blueprint service.status sensor containing result of /service/status * Updates due to upstream changes in etcd https://github.com/brooklyncentral/brooklyn-etcd/pull/11 * Updates mockito import `mockito-all`->`mockito-core` Change required due to upstream change https://github.com/apache/brooklyn-server/commit/5353ad01ef2972d04a69ff2cd7725280a013be99 * Updates BasicExternalConfigSupplierRegistry to use ClassLoaderUtils * Adds authSudo support (for sudo’ing with password) Required for some clouds (e.g. azure-arm), when using `useJcloudsSshInit: false` Tested on `azure-arm` using a location like that below (which previously failed): ``` brooklyn.catalog: id: azure-arm-lon-centos7 name: "Azure ARM UK South Centos 7.3" itemType: location item: type: jclouds:azurecompute-arm brooklyn.config: identity: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx credential: xxxxxxxx endpoint: https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx oauth.endpoint: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/oauth2/token jclouds.azurecompute.arm.publishers: OpenLogic jclouds.azurecompute.operation.timeout: 120000 jclouds.compute.resourcename-prefix: myprefix useJcloudsSshInit: false destroyOnFailure: false machineCreateAttempts: 1 logCredentials: true installDevUrandom: true osVersionRegex: 7.3 region: uksouth minRam: 2000 waitForSshable: 10m waitForConnectable: 10m user: myname ``` The reason this works is that `authSudo` is set in `JcloudsLocation` based on the jclouds `LoginCredentialsshouldAuthenticateSudo()`. jclouds gets this from a combination of the `jclouds.image.authenticate-sudo` property and the equivalent template option (https://github.com/jclouds/jclouds/blob/rel/jclouds-2.0.1/compute/src/main/java/org/jclouds/compute/options/TemplateOptions.java#L196). For azure-arm, jclouds hard-codes the default to true so it works out-of-the-box (see https://github.com/jclouds/jclouds-labs/blob/rel/jclouds-labs-2.0.1/azurecompute-arm/src/main/java/org/jclouds/azurecompute/arm/AzureComputeProviderMetadata.java#L96). * Add greaterThan and lessThan test framework assertions. Each relies on the two objects being compared being instances of the same `Comparable`. * Specify Java version for JBoss. JBoss does not work on Java 1.8 or above (see here: https://developer.jboss.org/message/808212) so specify 1.7 as opposed to falling back to the default. ## Known Issues Please note that known issues are logged in the Apache Brooklyn Jira available at https://issues.apache.org/jira/browse/BROOKLYN * BROOKLYN-491: IndexOutOfBoundsException parsing blobstore location definition "aws-s3" * BROOKLYN-487: Persistence to aws-ec2:eu-west-1 fails in jclouds, parsing "301 Moved Permanently" response (See https://issues.apache.org/jira/browse/JCLOUDS-1276) * BROOKLYN-486: Persistence to S3 ignores requested region. Persisting state to S3 ignores the specified region - it always uses us-east-1, automatically creating the bucket there. * BROOKLYN-480: AMP will accept invalid bundles in "brooklyn.libraries" Adding any jar file to brooklyn.libraries, even if not a bundle, will be accepted. * BROOKLYN-468: Restart effector does not restart the entity's feeds * BROOKLYN-459: BrooklynObjects replaced with null in Locations config on rebind. BrooklynObjects (excluding Entities, Locations, CatalogItems) set as config on Entities, Locations (and probably referenced anywhere else) get replaced by `null` on rebind. The reason is that the objects are deserialized (Mementos created) after constructing the entities, locations but not the rest of the BrooklynObjects. Therefore any non-location, non-entity `BrooklynObject` will be null. `BrooklynObject`s are treated separately from regular objects by replacing their value by an ID reference. * BROOKLYN-454: DSL config evaluation fails for inherited entity * BROOKLYN-450: AMP deployments fail on Azure ARM when VM hostname is changed Microsoft Azure Linux agent detects if machine's hostname is changed, initiates an update against the Azure cloud and restarts the machine's network scripts. It can lead to network disruptions and failing deployments, because of an inability for installing software packages from remote repositories and/or downloading of required assets from remote locations. You can observe messages like `Could not resolve host` in the error log. The proposed workaround is to configure the Azure Linux agent not to check for hostname changes and prevent the network restart. It can be achieved via changing the agent's configuration available in `/etc/waagent.conf` and setting the required parameter `Provisioning.MonitorHostName` from `y` to `n`. Azure Linux agent needs to be restarted after the change by executing its service script with `restart` option. Such restart for RedHat-based distributions (RHEL or CentOS) is possible with the following commands RedHat / CentOS 6 - `service waagent restart` RedHAt / CentOS 7 - `systemctl restart waagent` AMP can automate it either with [setup.script](https://docs.cloudsoft.io/locations/reference/#os-setup) or by using `pre.install.command` like ``` brooklyn.config: pre.install.command: | if [ -f /etc/waagent.conf ]; then sudo sed -i.bak 's/Provisioning\.MonitorHostName\=y/Provisioning\.MonitorHostName\=n/g' /etc/waagent.conf sudo service waagent restart fi ``` * The creation and attaching of volumes does not work in OpenStack Mitaka. See https://docs.cloudsoft.io/locations/misc/index.html#attaching-additional-volumes * The AMP UI has primarily been tested in Chrome and Firefox. These are the recommended browsers. In Safari, it has been observed that the quick launch "Add Config" does not work - there may be other unknown issues. For users of Internet Explorer, the recommendation is to use Chrome or Firefox instead; or failing that to upgrade to Microsoft Edge - but that is not officially supported. * On some environments, the `bin/client` tool for connecting to Karaf may see: IllegalStateException: Unable to negotiate key exchange for kex algorithms The client can be accessed alternatively by using `ssh -p 8101 karaf@localhost` (with default password being karaf). * The username to use when first logging into a new VM is sometimes inferred incorrectly. The solution is to explicitly specify the username in the location's configuration, e.g. using `loginUser: ubuntu`. For further information, see https://docs.cloudsoft.io/locations/reference/#os-initial-login-and-setup and https://docs.cloudsoft.io/operations/troubleshooting/deployment.html * When upgrading AMP, using existing persisted state, it will re-use the existing catalog from that persisted state and will not add new versions of the catalog items that ship with the upgraded AMP. As described in the "Upgrade Instructions", the workaround is to manually trigger the addition of those catalog items by running: br catalog add /opt/amp/catalog/catalog.bom * VM provisioning can fail, e.g. with the quick launch blueprints, when using the most basic location configuration. For example, the blueprint may require a particular OS distro, but the location picks an incompatible image, or the "login user" for initially ssh'ing to the VM may be incorrectly inferred for the chosen VM image. * The previous `./bin/amp copy-state` command is not available in Karaf. If required, an older version of AMP 3.x can be used to execute this command. * The previous `./bin/amp generate-password` command is not available in Karaf. If required, an older version of AMP 3.x can be used to execute this command. * There is no "reload properties" button in the new UI to reload the brooklyn.properties * The AMP log will occasionally report `NullPointerException` in `org.apache.cxf.transport.http.async`. These are not harmful and can be ignored. * Windows blueprints on vCloud Director sometimes fail. This is because the VM customizations performed by vCloud Director can include a reboot. This sometimes does not happen until several minutes after the VM is available, which can happen mid-way through installation of the software components. A workaround is available for this behaviour. Please check waitWindowsToStart parameter https://docs.cloudsoft.io/blueprints/base-blueprints/winrm/index.html#webserviceexception-could-not-send-message * It is strongly recommended to use WinRM v4 or greater. Older versions of WinRM (e.g. v3 on Windows 2008rc2) may fail. * For Windows VMs on Azure and GCE, the "Administrator" user is disabled. * BROOKLYN-243: On restarting an entity's process via the restart effector, there is a rare race-condition where the entity is not reported at "service.isUp" true, even though the process is running. * When using BYON to deploy multiple components to a single VM, the recommended way is to use the `SameServerEntity`. If instead the same machine were to be listed multiple times, then concurrent conflicting commands could be executed on that machine. For example, if multiple `iptables` commands are executed to add rules, then these interfere so that some rules may not be applied. * The following problems have been encountered in vCloud Director: * https://github.com/cloudsoft/jclouds-vcloud-director/issues/42 When rate-limited (getting a OPERATION_LIMITS_EXCEEDED response), the request is automatically retried with an exponential back-off. However, the command may have partially completed. This means an artifact with the given may have been partially created, causing a "duplicate name" error when retrying. * https://github.com/cloudsoft/jclouds-vcloud-director/issues/12 Failure terminating VM in vCloud Air: InvalidCacheLoadException when looking up the VM * https://github.com/cloudsoft/jclouds-vcloud-director/issues/23 VM delete failed; seemingly it did not wait for the stop to complete. * Windows provisioning to vCloud Air failed. The guest customization defined in the vApp Template appears not to be applied. The VM has no password, requires immediate password reset, and has not had the init script executed to configure WinRM. * Timeout waiting for VM to be provisioned, e.g. failing with a message like: "not composed within 1200000 ms" The following configuration can increase this timeout: brooklyn.location.named.MyVcdLocation.jclouds.compute.timeout.node-running=3600000 * The following are known issues with Azure Compute Classic: * JcloudsSecurityGroupCustomizer does not work with Azure, entities such as Riak or clocker will not work with this and a fixed, pre-configured network must be specified instead. * Private VM images are not supported. This is non-trivial to support, because private images and public images are modelled very differently in Azure. * Warning when deleting a VM. The error reported was: A disk with name ... is currently in use by virtual machine ... Those messages are to be considered warnings as they are produced by a retry logic that attempts to delete the disk while the VM is still being deleted. * Azure Compute Classic does not support port ranges for public endpoints; moreover it limits the number of endpoints per cloud service and per VM (150 per cloud service and 50 per VM). This causes provisioning to fail if a blueprint requires a large number of firewall rules (aka public endpoints). See https://social.msdn.microsoft.com/forums/azure/en-US/4c093e33-eb86-4639-b325-c295fa8bfc5d/vm-endpoint-and-port-range and https://feedback.azure.com/forums/216843-virtual-machines/suggestions/7120093-increase-150-endpoints-limit-for-a-cloud-service for more details. * The following are known issues with Azure ARM: * The following additional keys need to be specified in the location in order for it to work (substituting your choice of publisher and region): ``` jclouds.azurecompute.arm.publishers: OpenLogic region: southeastasia loginUser: amp templateOptions: overrideAuthenticateSudo: true ``` The publishers is any item from the list available here: https://docs.microsoft.com/en-us/azure/virtual-machines/virtual-machines-linux-cli-ps-findimage The region is any region from the list available here: https://azure.microsoft.com/en-us/regions/ The `loginUser` can be anything, as long as it's specified. * VM provisioning and deletion via jclouds can be very slow on ARM, particularly if there are many images to choose from and/or many storage accounts (i.e. many volumes). Setting a single publisher speeds this up considerably. * An issue has been identified with the upgrade to jclouds 2.0 which prevents the deployment of Windows blueprints to Azure Classic and Azure ARM. This issue only affects Windows blueprints * ARM rate limiting can introduce significant delays, particularly while performing concurrent actions. Rate limiting is applied at the account level, so different users of the same Azure account can cause limits to be applied to each other. For more information see the Azure ARM documentation here: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits One area which can improve multiple user access of Azure ARM is to increase the number of times AMP retries a request before failing. This can be set in the location or provisioning properties using the following key: jclouds.max-retries: 100 Also see https://issues.apache.org/jira/browse/JCLOUDS-1230 for a discussion of reducing the number of API calls. * In the persisted state, it is possible to have "dangling references" - e.g. the persisted state for an entity references another entity that does not exist (because that other entity is in the process of being created or deleted, for example). On startup, if the number of dangling references is too large then startup will fail. One can configure the tolerable number of dangling references. In the example below, each pair indicates for a given number of items how many dangling references are acceptable. This will be interpolated and extrapolated for other numbers of items. See the docs in https://github.com/apache/incubator-brooklyn/blob/0.8.0-incubating/utils/common/src/main/java/org/apache/brooklyn/util/collections/QuorumCheck.java#L87-#L112 For example, one could set in `/etc/amp/brooklyn.cfg`: rebind.failureMode.danglingRefs.minRequiredHealthy=[[0,0],[2,0],[3,1],[10,1],[100,10]] ## Backwards Compatibility This release only supports Java SE 8. For backwards compatibility with 3.x, see also the release notes for AMP 4.0.0. For backwards compatibility with 4.0.x or 4.1.x, see also the release notes for AMP 4.1.0 and 4.2.0. ## Upgrade Instructions See detailed upgrade instructions at http://docs.cloudsoft.io/operations/upgrade.html Note that from AMP 4.3 onward the RPM and DEB packages use the standard file system layout for persisted state and log files: - By default, persisted state is located at `/var/lib/amp`. - The `persistenceDir` and `persistenceLocation` are configured in the file `/etc/amp/org.apache.brooklyn.osgilauncher.cfg`. - The persistence details will be logged in `/var/log/amp/amp.info.log` at startup time. Also, for the tar.gz install the log is now in `./data/log/*` rather than `./log/*`. Note also that from AMP 4.3 onward only Java SE 8 is supported, therefore existing Java SE 7 installations should be upgraded to version 8. Use of RPM and DEB is now recommended, rather than the tar.gz. CentOS 7.x is recommended, though CentOS 6 and Ubuntu 16.04 are also tested. If binding to existing persisted state, an additional command is required to update the existing catalog with the latest AMP versions. Assuming AMP has been installed to /opt/amp (as is done by the RPM and DEB): br catalog add /opt/amp/catalog/catalog.bom