# AMP Release 4.6.0 * [Introduction](#introduction) * [Alert of Future Changes](#alert-of-future-changes) * [New Features](#new-features) * [Bug Fixes](#bug-fixes) * [Other Changes](#other-changes) * [Known Issues](#known-issues) * [Backwards Compatibility](#backwards-compatibility) * [Upgrade Instructions](#upgrade-instructions) ## Introduction Thank you to our community who have contributed a lot of improvements and feedback! Thanks also go to Brooklyn's commercial users who have funded this development and made some major contributions. For more information, please visit https://cloudsoft.io/amp, https://docs.cloudsoft.io and http://brooklyn.apache.org This is a minor upgrade from AMP version 4.5.X. ## Alert of Future Changes * Support for the Azure Classic (`jclouds:azurecompute`) location is now deprecated in favour of the newer Azure ARM (`jclouds:azurecompute-arm`). ## New Features #### AMP UI * Blueprint Composer UX overhaul - DSL relationships are now rendered on the graphical view - The sliding configuration view has been moved to the overview panel on the left - Each editable component (entity, policy, enricher and spec) is now represented on the graph - Items on the graph can now be clicked to edit / configure. - The item configuration section is now populated based on the available configuration keys for the selected entity. - Filters have been introduced to the configuration section - The graph shows better feedback in case of error. * Enrichers are now displayed in the catalog module #### AMP Network Security * AMP network security now supports Azure ARM #### Apache Brooklyn Server * Enrichers can now be managed via the Brooklyn catalog - Enrichers can be registerd using `@Catalog` on a class - Or in YAML by using `itemType: enricher` - API Updated with enricher endpoints - `GET /v1/catalog/enrichers` returns the list of all registered enrichers - `GET /v1/catalog/enrichers//` returns the detail of a specific enricher - `DELETE /v1/catalog/enrichers//` deletes a specific enricher * It is now possible to easily upload Bundles to AMP. - A bundle is a collection of YAML files and associated resources. A bundle is a zip or jar file that contains at the root a catalog.bom. This catalog bom is written using the catalog syntax and contains both bundle and version config. E.G.: brooklyn.catalog: bundle: MyServerBundle version: 1.0.0 items: - id: my-server item: type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess - Bundles can be sent to AMP using the BR command. This can be done with a zip/jar file (e.g. 'br catalog add bundle.jar'), or a folder (e.g. 'br catalog add bundleFolder'). When uploading a folder, the CLI will zip the folder first and then send it on. - Bundles can also be sent directly to the REST API by posting to the '/v1/catalog' endpoint. - Bundles are persited and are fully ready for production use. - Please note one known issue. It is not currently possible to post the same snapshot bundle multiple times. This will be fixed in a future release. * Virtual machine NICs in Azure ARM can now be configured * Azure ARM now uses managed disks (see [JCLOUDS-1257](https://issues.apache.org/jira/browse/JCLOUDS-1257)) * Support for Azure ARM custom images. To deploy a custom image stored in a resource group use imageId in format ResourceGroup/RegionYouDeploy/imageName. * Windows support for Azure ARM. http://docs.cloudsoft.io/locations/reference/index.html#using-windows-on-azure-arm See https://issues.apache.org/jira/browse/JCLOUDS-1301 ## Bug Fixes #### Apache Brooklyn Server * Fix JcloudsLocation config usage Ensure the configuration used for provisioning will correctly combine the location's configuration and the entity's `provisioning.properties`. Previously, for some config keys this was not correctly merged. * Fixed incorrect value being used when `WinRmMachineLocation.USE_HTTPS_WINRM` set - fixes [BROOKLYN-509](https://issues.apache.org/jira/browse/BROOKLYN-509) * Fixed type-coercion to/from arrays * Fixed type-coercion to Iterable * Fixed issue where certain sensitive fields loaded from external config suppliers no longer logged * Management plane id no longer changes on server restart - fixes [BROOKLYN-202](https://issues.apache.org/jira/browse/BROOKLYN-202) * WinRM: ensure passwords are not cached Previously when multiple BYON blueprints were submitted the NTLM session was cached in the winrm4j client. That means that if authentication is successful the underlying client will not try to obtain credentials supplied for location. That is solved by reusing winrm4j cxf context for each connection. https://github.com/cloudsoft/winrm4j/issues/63 * Fixed [BROOKLYN-515] WindowsPerformanceCounterSensors does not show values on some Windows deployments * Additional disks added by the `addExtraHdd` effector or by `NewVolumeCustomizer` will now be deleted on expunge #### Apache Brooklyn Library * Update NGiNX dependencies On some clouds, notably Azure with CentOS, `kernel-headers` is in the disbaled list which causes `gcc` to not install properly. * CouchDB config updated for version `2.0.0`, now uses the httpd section for binding ip and setting port * BROOKLYN-516 WindowsPerformanceCounterSensors does not start on rebind #### Container Service * Fixed issue where provided config would not be used Provided config to a k8s entity would be ignored and the default value used instead * Updated KubernetesPod entity Improves handling of Kubernetes pods and resources. - Change `KubernetesPod` entity into an extension of `DockerContainer`, rather than a placeholder. - Rationalise configuration by splitting keys between location and entity. - Add new `KubernetesMachineLocation` with SSHable and empty variants for management by the `KubernetesLocation`. - Add `OpenShiftPod` and `OpenShiftResource` entities that extend Kubernetes versions. - Create catalog entries in BOM file for new entities. * Kubernetes Updates - Use KubernetesPod as entity for container deployment - Updates OpenShift so that it extends Kubernetes - Set pod names based on configuration - Handle multiple addresses for services and set resource type correctly #### Blueprint :: Clocker * Updated `brooklyn-dns` to version 0.1.3 which fixed concurrency issues #### Blueprint :: Elastic Stack (ELK) * Update bind address to `0.0.0.0` by default Also changes config of elasticsearch to add `elasticsearch.publish.address` And semantics of bind.address so that the latter only sets the bind address (with publish.address defaulting to the subnet ip). Necessary on Softlayer, where a VM has different nics for the subnet and the public addresses. #### Blueprint :: Jenkins * Update bind address to `0.0.0.0` by default Necessary on Softlayer, where a VM has different nics for the subnet and the public addresses. ## Other Changes #### Apache Brooklyn Server * BROOKLYN-513: Improved support for complex location configuration, passed to jclouds. Complex types that are used in jclouds configuration can now be defined in yaml blueprints. * BROOKLYN-510 :: Fix for inboundPorts in certain clouds not getting opened due to firewalld * Usage of GroovyJavaMethods.* has been removed Motivated by encountering more problems like that in https://issues.apache.org/jira/browse/BROOKLYN-449, but this time when it tried to call `DslComponent` which called `GroovyJavaMethods.truth`. Now the only use of `GroovyJavaMethods` are: * If we genuinely have a Groovy-specific object, such as a `Closure` * `AbstractAggregatingEnricher` constructor uses `GroovyJavaMethods.castToPredicate()` --- These changes will change the semantics of our `attributeWhenReady` etc very slightly, for people taking advantage of the more unusual groovy'isms. The difference between `GroovyJavaMethods.truth` and `JavaGroovyEquivalents.groovyTruth()` is that the former also handles "matchers" and classes with an asBoolean method (see http://groovy-lang.org/semantics.html#Groovy-Truth). * BROOKLYN-449 remove usage of Groovy for effector method calls * Adds configuration flag to allow a Winrm4jTool to use a unique context To enable add the following field in a location configuration: `brooklyn.winrm.config.use.unique.context: true` * Adds JcloudsLocationCustomizer hook to allow node / config configuration Can be used by a customizer to configure the ssh / winrm config tool with runtime data such as machine id * Added EBS Location Customizers Allows user to specify an additional AWS hard drive to be added at provisioning time, e.g.: provisioning.properties: customizers: - $brooklyn:object: type: org.apache.brooklyn.location.jclouds.aws.EbsVolumeCustomizers$WithNewVolume object.fields: availabilityZone: us-west-1a ec2DeviceName: /dev/sdf osDeviceName: /dev/xvdf mountPoint: /data permissions: 700 sizeInGib: 20 filesystemType: ext4 deleteOnTermination: false * Delete hazelcast datagrid (and delete datagrid config) Use of the hazelcast datagrid was considered experimental, and was not supported for production use-cases * Minor CatalogResource and ApplicationResource fixes CatalogResource closes ZipFiles cleanly and ApplicationResource checks the invoke-effector permission before invoking the start effector. * Adds resource copy for customize phase #### Apache Brooklyn CLI * Display bundle info & message upon `br catalog add ...` - supports backwards compatibility as only some API methods support the detail * Delete request will now only fail if an unsuccessful code is returned previously the request would fail if anything other that 204 was returned * Remove deprecated `reset-catalog` command - API has been removed upstream #### Apache Brooklyn REST Client * Brooklyn REST API convenience has better error propagation Previously the convenience `getEntity` method masked errors the way some clients used it. Clarify method to fail on the most obvious errors (instead of returning a totally useless object of the type requested), and introduce a better method. #### Brooklyn Dist * Enable TLS 1.1 & 1.2 explicitly If not explicitly set java could drop back to an older now unsupported version of TLS #### AMP * Removed Old JS GUI #### AMP Jumphost * Adds support for plugging in message handler customizer(s). These hooks allow one to perform specific customisation of the configuration used for ssh and WinRM connections and operations. For example, an implementation could inject credentials, do additional logging, etc. #### AMP Networking * Added AMP networking support for Azure ARM #### AMP Node * Added ability to append to system properties * Fixed issue where default admin password would not be removed sed command was missing sudo so lacked privileges #### Alien4Cloud * Alien4Cloud UI support has been removed ## Known Issues #### Azure ARM Orphaned Resources * Under certain conditions when a VM is provisioned and removed or when a VM fails to provision, the associated resources are not removed. The current workaround requires manually deleting the orphaned resources - Resource Group - Virtual Network - Network Security Group - Network Interfaces - Disks - Availability Sets #### Azure ARM Networking * The Azure ARM network implementation has changed such that each VM get's it's own subnet If you are deploying an application and require that VMs can communicate with each other on private IPs then you must either - Use [AMP network security](https://docs.cloudsoft.io/blueprints/yaml-syntax/networking/index.html) - Manually create a network within Azure (add the below config to your location template options) templateOptions: ipOptions: - allocateNewPublicIp: true subnet: /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks//subnets/default #### Azure ARM Windows * AMP-1778 If a windows machine has been deployed to Azure ARM using AMP 4.6.0, previous versions of AMP will no longer be able to deploy to Azure until the machine is removed #### Security Group Caching * When a security group used by AMP is deleted externally and subsequently AMP tries to delete it and recreate it again it will fail, still using a stale identifier for the group. Affects the `NetworkSecurityCustomizer` and its usage of shared security groups. Tracked in https://issues.apache.org/jira/browse/JCLOUDS-1307. ## Backwards Compatibility * This release only supports Java SE 8. * For backwards compatibility with 3.x, see also the release notes for AMP 4.0.0. * For backwards compatibility with 4.x please see the intermediate release notes * Support for deprecated `~/.brooklyn/catalog.xml` has been removed Use of `~/.brooklyn/catalog.xml` has been deprecated since Brooklyn 0.7.0. - Note that it deletes the rest api endpoint `/v1/catalog/reset` (which relied on there being a `catalog.xml`). - It also changes the way `fragment` is handled in things like `/v1/catalog/applications?fragment=...`. Previously it was use the `toXml`, and then do a case-insensitive search for the fragment. To try to keep the functionality as close as possible (with code as simple as possible), I've reused the persisted state serialization mechanism to produce the memento, and then do the case-insensitive search for the fragment in that. ## Upgrade Instructions * See detailed upgrade instructions at https://docs.cloudsoft.io/operations/upgrade.html * Note that from AMP 4.3 Java SE 8 is required * Use of RPM and DEB is now recommended, rather than the tar.gz. * CentOS 7.x is recommended, though CentOS 6 and Ubuntu 16.04 are also tested. * If binding to existing persisted state, an additional command is required to update the existing catalog with the latest AMP versions. Assuming AMP has been installed to /opt/amp (as is done by the RPM and DEB): br add-catalog /opt/amp/catalog/catalog.bom