AMP Release 4.0.1 * Introduction * Alert of Future Changes * New Features * Bug Fixes * Known Issues * Backwards Compatibility * Upgrade Instructions ### Introduction Thanks you to our community who have contributed a lot of improvements and feedback! Thanks also go to Brooklyn's commercial users who have funded this development and made some major contributions. For more information, please visit http://www.cloudsoft.io/amp, http://docs.cloudsoft.io and http://brooklyn.apache.org This is a minor upgrade from AMP version 4.0.0. ### Alert of Future Changes * Longer term, the persistence naming will be changed to avoid using package and class names in persisted state. This will improve backwards compatibility support moving forwards. A transformation tool will be provided for this update. * Brooklyn networking is being revisited, to incorporate work pioneered in the Clocker and Calico projects. This will allow networking requirements to be more easily described within an application, and the appropriate networking configured (be it VPCs, subnets, security groups or NAT'ing). There is a proposal at https://docs.google.com/document/d/1IrWLWunWSl_ScwY3MRICped8eJMjQEH1FbWZJcoK0Iw for the handling of multiple networks. This lays out a change in the semantics of sensors, to make it clearer which network an ip/port applies to. * The configuration options for both Azure and WinRM (especially for password and use of NTLM) are under review, and may be changed in a future release. Please treat these configuration options as "beta". * Code that was already deprecated in AMP 2.x and AMP 3.x will be deleted (incrementally). Additional code that has been deprecated will be deleted in a future release, in line with out deprecation policy. ### New Features * AMP-1193 Tosca support has been re-enabled. For further details, see http://docs.cloudsoft.io/operations/alien4cloud/index.html There is a known issue where an additional `bin/stop` and `bin/start` may be required after running the `feature:install` for the Tosca karaf feature. * AMP-1238 OpenStack Mitaka is now supported. This requires using the location type "jclouds:openstack-mitaka-nova", rather than the vanilla "jclouds:openstack-nova". The location type "jclouds:openstack-devtest-compute" has been removed; this new Mitaka location type should be used instead. * AMP-1233 Deploying of Windows blueprints (via WinRM) in Karaf is re-enabled. For further details, see http://docs.cloudsoft.io/blueprints/base-blueprints/winrm/index.html ### Bug Fixes * AMP-1206 Modifying etc/org.apache.brooklyn.osgilauncher.cfg will no longer cause the AMP management context to restart. Instead, the AMP service must be explicitly restarted for changes to that file to take effect. * AMP-1232 Java-based blueprints on Ubuntu 16.04 now work. Previously the Java install failed (it attempted to install Java 7, which is not available). Now it will attempt a Java 7 install, and then fallback to a Java 8 install. This affects stock blueprints like Tomcat, JBoss AS7, etc. * AMP-1230 The default minRam for VM provisioning is now 1G. Previously, there was no default minRam. If no hardware preferences were set, then on some clouds it would pick the smallest VM possible. For example, on GCE it would use "f1-micro" with 614MB of RAM, which meant that many blueprints would fail with these default settings. * AMP-1215 The config option "openIptables" default value has been changed to "true" for jclouds locations. This means that iptables and/or firewalld rules will be updated to open the same ports as would be opened in the cloud's security group. * AMP-1240 The AMP QA Test Framework blueprint (for provisioning a new AMP QA Framework) has two important fixes: * The AMP version installed by the blueprint will now be the latest version available. * It now defaults to CentOS 7 for the AMP and Dashboard components. Previously it was necessary to configure this in the location configuration options. ### Known Issues * AMP-1254 Alien4Cloud app deployment fails when an app is composed in the A4C UI. * AMP-1249 After installing the Tosca feature in Karaf, an additional `bin/stop` and `bin/start` may be required. * AMP-1248 On some environments, the `bin/client` tool for connecting to Karaf may see: IllegalStateException: Unable to negotiate key exchange for kex algorithms The client can be accessed alternatively by using `ssh -p 8101 karaf@localhost` (with default password being karaf). * AMP-1250 The latest OpenStack (Mitaka) requires use of the location type `openstack-mitaka-nova`, rather than the vanilla `openstack-nova`. * AMP-1226 The username to use when first logging into a new VM is sometimes inferred incorrectly. The solution is to explicitly specify the username in the location's configuration, e.g. using `loginUser: ubuntu`. For further information, see http://docs.cloudsoft.io/locations/reference/#os-initial-login-and-setup and http://docs.cloudsoft.io/operations/troubleshooting/deployment.html * AMP-1213 When upgrading AMP, using existing persisted state, it will re-use the existing catalog from that persisted state and will not add new versions of the catalog items that ship with the upgraded AMP. As described in the "Upgrade Instructions", the workaround is to manually trigger the addition of those catalog items by running: br add-catalog /opt/amp/catalog/catalog.bom * AMP-1202 The RPM install does not work on CentOS 6.x. Instead, CentOS 7.x is recommended. * AMP-1215,AMP-1226 VM provisioning can fail, e.g. with the quick launch blueprints, when using the most basic location configuration. For example, the blueprint may require a particular OS distro, but the location picks an incompatible image, or the "login user" for initially ssh'ing to the VM may be incorrectly inferred for the chosen VM image. * AMP-1220 The default logging configuration writes the jclouds wire log to log/amp.debug.log. If a lot of calls are made to the cloud (particularly if using S3 as the persistence store), this can quickly fill the debug log. In a future release, the logging will be split into separate files. * AMP-1221 The "VM customization" module is missing from AMP 4.0. The main implication is that the VcloudDirectorImageChooser is not included. * AMP-1227 Use of the AMP jumphost for ssh command execution has problems, due to issues with the correct classloading in Karaf. * AMP-1235 When using an empty brooklyn.properties, if the "reload brooklyn.properties" is used (either in the classic UI or via the REST api), it reloads the HTTP login credentials. If no credentials are defined in brooklyn.properties, then this means the original default credentials will no longer work. * AMP-877 the previous `./bin/amp copy-state` command is not available in Karaf. If required, an older version of AMP 3.x can be used to execute this command. * AMP-1043 the previous `./bin/amp cloud-compute` command is not available in Karaf. If required, an older version of AMP 3.x can be used to execute this command. * AMP-883 the previous `./bin/amp generate-password` command is not available in Karaf. If required, an older version of AMP 3.x can be used to execute this command. * The new UI has some features missing that were present in the classic UI. The workaround is to access these via the classic UI, or the `br` CLI, or the REST api: * There is no "reload properties" button, to reload the brooklyn.properties * No "expunge" and "unmanage" button, to complete remove an entity. Calling the "stop" effector on a top-level app with attempt to gracefully stop and unmanage the entity, but if stop fails then unmanage will not be done. * AMP will occasionally report NullPointerException in org.apache.cxf.transport.http.async. These are not harmful and can be ignored. * BROOKLYN-317 VM provisioning can fail in AWS-EC2 (because api did not return public IP). The workaround is to configure machineCreateAttempts on the location, setting it to a number greater than 1. It will thus retry if the problem is encountered. * Windows blueprints on vCloud Director sometimes fail. This is because the VM customizations performed by vCloud Director can include a reboot. This sometimes does not happen until several minutes after the VM is available, which can happen mid-way through installation of the software components. A workaround is available for this behaviour. Please check waitWindowsToStart parameter http://docs.cloudsoft.io/blueprints/base-blueprints/winrm/index.html#webserviceexception-could-not-send-message * It is strongly recommended to use WinRM v4 or greater. Older versions of WinRM (e.g. v3 on Windows 2008rc2) may fail. See COMPOSEC-1200. * For Windows VMs on Azure and GCE, the "Administrator" user is disabled. * https://issues.apache.org/jira/browse/BROOKLYN-243: On restarting an entity's process via the restart effector, there is a rare race-condition where the entity is not reported at "service.isUp" true, even though the process is running. * When using BYON to deploy multiple components to a single VM, the recommended way is to use the `SameServerEntity`. If instead the same machine were to be listed multiple times, then concurrent conflicting commands could be executed on that machine. For example, if multiple `iptables` commands are executed to add rules, then these interfere so that some rules may not be applied. * The following problems have been encountered in vCloud Director: * https://github.com/cloudsoft/jclouds-vcloud-director/issues/12 Failure terminating VM in vCloud Air: InvalidCacheLoadException when looking up the VM * https://github.com/cloudsoft/jclouds-vcloud-director/issues/23 VM delete failed; seemingly it did not wait for the stop to complete. * Windows provisioning to vCloud Air failed. The guest customization defined in the vApp Template appears not to be applied. The VM has no password, requires immediate password reset, and has not had the init script executed to configure WinRM. * Timeout waiting for VM to be provisioned, e.g. failing with a message like: "not composed within 1200000 ms" The following configuration can increase this timeout: brooklyn.location.named.MyCanopyLocation.jclouds.compute.timeout.node-running=3600000 * The following are known issues with jclouds-azure: * Private VM images are not supported. This is non-trivial to support, because private images and public images are modelled very differently in Azure. * The classic REST API is used, rather than the new Azure Resource Manager REST API. This classic API will likely be deprecated; Microsoft encourage the use of the new API. * Warning when deleting a VM. The error reported was: "A disk with name ... is currently in use by virtual machine ..." Those messages are to be considered warnings as they are produced by a retry logic that attempts to delete the disk while the vm is still being deleted. * Azure does not support port ranges for public endpoints; moreover Azure limits the number of endpoints per cloud service and per VM (150 per cloud service and 50 per VM). This causes provisioning to fail if a blueprint requires a large number of firewall rules (aka public endpoints). See https://social.msdn.microsoft.com/forums/azure/en-US/4c093e33-eb86-4639-b325-c295fa8bfc5d/vm-endpoint-and-port-range and https://feedback.azure.com/forums/216843-virtual-machines/suggestions/7120093-increase-150-endpoints-limit-for-a-cloud-service for more details * In the persisted state, it is possible to have "dangling references" - e.g. the persisted state for an entity references another entity that does not exist (because that other entity is in the process of being created or deleted, for example). On startup, if the number of dangling references is too large then startup will fail. One can configure the tolerable number of dangling references. In the example below, each pair indicates for a given number of items how many dangling references are acceptable. This will be interpolated and extrapolated for other numbers of items. See the docs in https://github.com/apache/incubator-brooklyn/blob/0.8.0-incubating/utils/common/src/main/java/org/apache/brooklyn/util/collections/QuorumCheck.java#L87-#L112 For example, one could set in brooklyn.properties: rebind.failureMode.danglingRefs.minRequiredHealthy=[[0,0],[2,0],[3,1],[10,1],[100,10]] ### Backwards Compatibility OpenStack Mitaka is now supported. The previous workaround was to use the location type "jclouds:openstack-devtest-compute". This has been removed. Instead, "jclouds:openstack-mitaka-nova" should be used. Any blueprints deployed with the old "openstack-devtest-compute" may fail when attempting to stop VMs, or start additional VMs. The workaround is to modify the persisted state to instead refer to "openstack-mitaka-nova" - contact Cloudsoft Support for further details. The default value for the configuration option "minRam" has been changed to 1G. Previously there was no default, so on some clouds the smallest possible hardware profile would be used. The default value for the configuration option "openIptables" has been changed to "true". This can be easily disabled by adding to the location config `openIptables: false`. ### Upgrade Instructions Use of RPM and DEB is now recommended, rather than the tar.gz. CentOS 7.x is recommended over CentOS 6.x (indeed the RPM will not work on CentOS 6.x). This entirely replaces the previous install. If binding to existing persisted state, an additional command is required to update the existing catalog with the AMP 4.x versions. Assuming AMP has been installed to /opt/amp (as is done by the RPM and DEB): br add-catalog /opt/amp/catalog/catalog.bom All existing custom jars previously added to lib/plugins (e.g. for Java-based entities) need to be converted to OSGi bundles, and installed in Karaf. The use of the "brooklyn.libraries" section in catalog.bom files will continue to work. User credentials should now be recorded in ${karaf.home}/etc/brooklyn.cfg. AMP will still read them from both ${karaf.home}/etc/brooklyn.cfg and ~/.brooklyn/brooklyn.properties