vFabric GemFire 7.0.0 Release Notes

What's in the Release Notes

The vFabric GemFire 7.0.0 release notes cover the following topics:

• What's New in vFabric GemFire 7.0
• Upgrading to vFabric GemFire 7.0
• Map of Old Commands to New gfsh Commands in vFabric GemFire 7.0
• Resolved Issues
• Known Issues

What's New in vFabric GemFire 7.0

vFabric GemFire 7.0 includes the following feature changes and enhancements:

• New WAN API with Parallel WAN and Simplified WAN Configuration
• Deprecation Notes for Earlier WAN API
• Main Steps to Upgrade Existing Deployments to Use the New WAN API
• Redesigned GemFire Management and Monitoring System
• Unified GemFire SHell ('gfsh') Command-Line Interface
• New GemFire Pulse Monitoring Tool
• Enhanced Usability and Performance for Disk Stores and Persistence
• Ability to Import and Export Cache and Region Snapshots
• Ablility to Deploy and Undeploy JARs Dynamically
• Support for Logical Member Groups and Function Execution on Member Groups
• Added Consistency Checking for Distributed Regions
• New Parallel Write-Back Cache Listeners and Simplified Listener Configuration
• Improved Querying Performance
• Support for Adding and Retrieving JSON Documents in vFabric GemFire
• Native Support for Memcached Clients with Gemcached
• Better Release Integration with Spring Data GemFire
• Added GemFire Optimization Guidelines and Certification on VMware vSphere
• Enhanced Integration for vFabric GemFire HTTP Session Management Module for AppServers
• Improvements to Product Packaging and Distribution
• Changes to Product Documentation

New WAN API with Parallel WAN and Simplified WAN Configuration

vFabric GemFire 7.0 simplifies configuration for WAN deployments, and introduces linear scaling for distributing region events across a WAN. You configure WAN deployments by defining one or more gateway senders in distributed systems that propagate events over a WAN, and by defining gateway receivers in systems that receive those events. See Configuring a Multi-site (WAN) System in the vFabric GemFire User's Guide for more information.

The following enhancements and changes apply to deployments that use the new WAN API. (For usage information about the earlier WAN API, see Deprecation Notes for Earlier WAN API.)

• Parallel gateway senders for highest throughput. Enables multiple partitions to propagate events synchronously to a remote WAN destination. As you add GemFire members to host additional partitions, you increase throughput for WAN distribution by deploying additional parallel gateway senders on those members. Each parallel gateway sender preserves ordering only for the data partition that it manages. If you must preserve event ordering for the region as a whole you can use a serial gateway sender, at the expense of providing less throughput for WAN distribution. See How Multi-site (WAN) Systems Work and Configuring Dispatcher Threads and Order Policy for Event Distribution in the vFabric GemFire User's Guide for more information.
• User-defined filters for event handling. GemFire 7.0 deployments that use the new WAN API can deploy user-defined filters to determine which events are distributed over a WAN, and to compress or encrypt the events that are distributed. See Filtering Events for Multi-Site (WAN) Distribution in the vFabric GemFire User's Guide for more information.
• Eventual consistency for distributed region updates. By default, all GemFire WAN deployments use Gemfire 7.0's region consistency features to ensure that regions that are distributed over a WAN eventually reach a consistent state. The default consistency implementation for WAN distribution uses a timestamp mechanism to determine whether a region update from a new GemFire site should be applied in the local system. Alternatively, you can deploy a user-defined conflict resolver to determine whether to apply a WAN event that could cause inconsistencies in a region. See Consistency for Region Updates and Resolving Conflicting Events in the vFabric GemFire User's Guide for more information.

Deprecation Notes for Earlier WAN API

• All elements of the previous WAN API deprecated. These elements include the gateway-hub, gateway, gateway-endpoint, gateway-queue, and gateway-listener elements in cache.xml, as well as the GatewayHub, Gateway, GatewayQueueAttributes, AttributesFactory.setEnableGateway(), RegionFactory.setGatewayHubId APIs.
• Existing WAN deployments are supported by the deprecated WAN API. vFabric GemFire 7.0, 6.6, and 6.5 sites can communicate over a WAN using the deprecated API. Parallel distribution, filters, and conflict resolvers are available only with the new GemFire 7.0 WAN API.

Main Steps to Upgrade Existing Deployments to Use the New WAN API

To upgrade a GemFire deployment to use the WAN API, first update the GemFire software to version 7.0 according to Upgrading vFabric GemFire in the vFabric GemFire User's Guide. Then follow these guidelines for upgrading the WAN configuration. For more details, see Configuring a Multi-site (WAN) System in the vFabric GemFire User's Guide.

1. Stop all operations in each vFabric GemFire cluster that participates in the WAN configuration.
2. After all events are drained from gateway queues, and the distributed WAN events are applied, shut down each GemFire cluster.
3. Upgrade the WAN configuration in members of each cluster.
   
   

   Ensure that each cluster uses locators for member discovery. WAN deployments are not supported when using multicast discovery. In the configuration for each locator:

   • Use the distributed-system-id property to define a unique ID for the local cluster.
   • Use the remote-locators property to identify locators in remote clusters to which the local cluster will connect.

   For each cluster that applies updates received over a WAN:

   • Configure a gateway receiver in each member that will apply WAN events.
   • Ensure that each member that hosts a gateway receiver also hosts every region for which it might receive an update event over the WAN.

   For each cluster that will distribute region updates to another cluster over a WAN:

   • Create one or more gateway sender configurations in each member that will distribute region events over the WAN.
   • Configure regions to use one or more gateway senders to distribute their events to remote clusters.
   • Ensure that each parallel gateway sender configuration is depoyed to all members that host a region that uses the sender configuration.

4. Restart each GemFire cluster that will apply WAN-distributed events.
5. Restart each GemFire cluster that will distribute region events over the WAN.

Redesigned GemFire Management and Monitoring System

vFabric GemFire 7.0 greatly simplifies the management and monitoring of vFabric GemFire distributed systems. Unlike the previous management and monitoring system, GemFire 7.0 integrates monitoring directly within GemFire processes, and adds the ability to manage these processes using the same system. You can now capture metrics directly from managed nodes, without needing to install and configure a separate monitoring process. The new GemFire management and monitoring system uses a federated Open MBean strategy to propagate information member states and system health to a centralized MBean Server.

Here are some highlights of the redesigned system:

• All GemFire MBeans are now MXBeans, which provide an easier-to-use and more scalable method for monitoring vFabric GemFire. In addition, all GemFire members are directly manageable through their MXBeans and individually hosted Platform MBean Server.
• The JMX Agent, which required separate installation and configuration, is now deprecated.
• The com.gemstone.gemfire.admin API is now deprecated and replaced by the new management API. See the Java API documentation for com.gemstone.gemfire.management for detailed API additions.
• Because the new management and monitoring system uses JMX, you can plug-in any standard JMX-compliant management or monitoring tool, such as JConsole or jvisualvm to inspect any GemFire node.

DataBrowser and Visual Statistics Display (VSD) are modified to work with the new monitoring and management components. Both tools are bundled with the vFabric GemFire product, and no longer require a separate installation and download.

vFabric GemFire 7.0 also bundles new monitoring and management interfaces. See the gfsh and GemFire Pulse sections of this document for an overview of these tools.

Unified GemFire SHell ('gfsh') Command-Line Interface

The completely revised GemFire SHell ('gfsh') is a major usability improvement. Use the intuitive gfsh command-line interface to start and stop GemFire servers and locators; configure members; view metrics; create and populate regions; deploy JARs dynamically across your members; detect and debug deadlocks; manage disk-stores; and import and export data snapshots.

Inspired by Spring roo, gfsh provides command completion, context-sensitive help, and scripting. It also incorporates functionality from previous GemFire utilities and scripts such as gemfire, cacheserver, and gemfire start/stop locator.) See Using the GemFire SHell ('gfsh') in the vFabric GemFire User's Guide for more information.

Note: The gemfire and cacheserver scripts located in the <product_dir>/bin directory are now deprecated. The scripts are still included with the product for backwards-compatibility support. For a mapping of the old commands to the new ones in gfsh, see Map of Old Commands to gfsh Commands in vFabric GemFire 7.0.

New GemFire Pulse Monitoring Tool

GemFire Pulse is a new Web browser-based monitoring dashboard that allows you to view and monitor the current status of your entire GemFire deployment. You can view an overall dashboard of distributed system health, and then drill down into viewing members and regions. GemFire Pulse replaces the GFMon tool. See GemFire Pulse in the vFabric GemFire User's Guide for more information.

Enhanced Usability and Performance for Disk Stores and Persistence

Many improvements to disk stores and persistence are tied to new gfsh commands through which you can operate and manage disk-stores.

• Incremental backup option for GemFire disk-stores. Requires fewer system resources for disk writes during a backup. You can recover systems from the last full backup or specify an incremental backup for recovery. See Back Up and Restore a Disk Store in the vFabric GemFire User's Guide for more information.
• New disk store commands. Modify disk-store configurations (alter disk-store), discover and revoke missing disk-stores (show missing-disk-stores, revoke missing-disk-stores).
• Disk converter utility to upgrade disk-stores from GemFire 6.5 or 6.6 to 7.0. See the upgrade offline disk-store command.

Ability to Import and Export Cache and Region Snapshots

You can now import and export cache and region data snapshots by using the new RegionSnapshotService or CacheSnapshotService APIs. Exporting and importing data is useful for recovering system data or for moving data quickly from system to another system. You can save snapshots of any type of region as well as entire caches. With the previous API, you could only save snapshots of replicated regions. See Cache and Region Snapshots in the vFabric GemFire User's Guide for more information and the Java API documentation for RegionSnapshotService and CacheSnapshotService.

The RegionSnapshotService and CacheSnapshotService APIs replace the previous Region.loadSnapshot and Region.saveSnapshot APIs, which are now deprecated.

Ability to Deploy and Undeploy JARs Dynamically

You can now dynamically deploy your single or multiple application JAR files to specific members or to all members in your distributed system. GemFire automatically keeps track of JAR file versions and provides conveniences such as autoloading the deployed JAR files to the CLASSPATH and auto-registering any functions that are contained within the JAR.

To deploy and undeploy application JAR files in vFabric GemFire, use the gfsh deploy or undeploy command. You can deploy a single JAR or multiple JARs (by either specifying the JAR filenames or by specifying a directory that contains the JAR files), and you can also target the deployment to a member group or multiple member group.

Support for Logical Member Groups and Function Execution on Member Groups

Member groups replace "server groups," a way of grouping servers that was only visible to clients. Now you can define logical member groups that are recognizable throughout the distributed system. You can target groups for specific management operations such as backup operations or JAR application deployment. Using logical member groups can help you achieve parallelism in your management and operation of distributed system members. See Using Member Groups in the vFabric GemFire User's Guide for more information.

With this feature, you can also target specific member groups or single members within a member group when executing functions. See How Function Execution Works in the vFabric GemFire User's Guide for more information and FunctionService in the Java API documentation.

The group definition in server-side cache.xml is now deprecated. Use the groups property in gemfire.properties instead. Additionally, any existing roles that you have defined in your gemfire.properties file are now considered member groups.

Added Consistency Checking for Distributed Regions

Consistency checking ensures that all copies of a distributed region eventually reach a consistent state on all members and clients that host the region, including GemFire members that apply region events across a WAN. Unlike previous versions of GemFire, GemFire 7.0 members detect and reject an out-of-order update to a region entry when consistency checking is enabled. If a concurrent update is detected, GemFire 7.0 members choose only one of the two updates to apply. Potentially conflicting WAN events are resolved using a timestamp by default, or by using an optional conflict resolver that you define.

With consistency checks enabled, GemFire 7.0 regions require additional overhead (as compared to GemFire 6.x) for storing version and timestamp information, and for retaining deleted entries for a period of time to ensure consistency. See "Overhead for Consistency Checks" in Consistency for Region Updates in the vFabric GemFire User's Guide for more information.

New Parallel Write-Back Cache Listeners and Simplified Listener Configuration

vFabric GemFire 7.0 simplifies the configuration API for installing write-back cache listeners, and introduces parallel write-back cache listeners for linear scaling in distributing region events to listener implementations. Parallel queues are deployed to multiple GemFire members along with their listener implementations, and each queue and listener processes all region events for that local member. See Implementing an AsyncEventListener for Write-Behind Cache Event Handling in the vFabric GemFire User's Guide for more information.

GemFire 7.0 deprecates the GatewayEventListener, GatewayHub, and Gatwey APIs used to configure write-back cache listeners in earlier versions of the product.

Improved Querying Performance

Enhancements enable you to improve querying performance as follows:

• Create and query hash-based indexes by using APIs or cache.xml. Creating a hash index improves the memory usage of your index. See Creating Hash Indexes in the vFabric GemFire User's Guide for more information.
• Monitor queries for low memory conditions and prevent out-of-memory exceptions. A new resource management feature gives you more control over memory issues. See Monitoring Queries for Low Memory in the vFabric GemFire User's Guide for more information.
• Detect continuous querying (CQ) connections and disconnections. The CqStatusListener interface lets you implement a listener that detects when a CQ is connected or disconnected from a server. See Implementing Continuous Querying in the vFabric GemFire User's Guide for more information.
• Return all durable CQs. The QueryService API method from getAllDurableCqsFromServer returns all the durable CQs registered by the client calling the method. See QueryService in the Java API documentation for more information.
• Improve general query performance. Bug fixes to improve overall query speed, lower memory footprints, and remove contention issues in querying.

Support for Adding and Retrieving JSON Documents in vFabric GemFire

vFabric GemFire now supports the use of JSON-formatted documents natively. When you add a JSON document to a GemFire cache, you call the JSONFormatter APIs to transform it into the PDX format (as a PdxInstance), which allows GemFire to understand the JSON document at a field level. You can perform the same GemFire operations on the JSON document that you can perform on a PdxInstance. See Adding JSON Documents into the GemFire Cache in the vFabric GemFire User's Guide for more information.

Note: This API is not yet available in vFabric GemFire native client.

Native Support for Memcached Clients with Gemcached

Gemcached is a light-weight adapter that allows memcached clients to connect to GemFire. A Gemcached server is embedded within a GemFire cache server and listens for connections from memcached clients. The Gemcached server translates the memcached protocol to use GemFire APIs. Memcached clients can be written in many programming languages and do not need to be modified. Gemcached allows memcached clients to connect to any Gemcached server, which gives memcached clients access to GemFire features such as replication and high-availability. See Gemcached in the vFabric GemFire User's Guide for more information.

Better Release Integration with Spring Data GemFire

Starting with vFabric GemFire 7.0, vFabric GemFire and Spring Data GemFire releases will now be coordinated. vFabric GemFire 7.0 is supported by Spring Data GemFire 1.2.1, which is publicly available at http://www.springsource.org/spring-gemfire.

In addition, Spring Data GemFire documentation has been added to the vFabric GemFire User's Guide. See Getting Started with Spring Data GemFire in the vFabric GemFire User's Guide.

For more information on new features in Spring Data GemFire, see the Spring Data GemFire Reference Guide: http://static.springsource.org/spring-data/gemfire/docs/current/reference/html/index.html.

Added GemFire Optimization Guidelines and Certification on VMware vSphere

GemFire 7.0 is performance-tested and certified to run optimally on VMware vSphere 5.0. The vFabric GemFire User's Guide for more information publishes basic guidelines on how to optimize vSphere-based installations of GemFire. See Improving GemFire Performance on vSphere in the vFabric GemFire User's Guide for more information.

Enhanced Integration for vFabric GemFire HTTP Session Management Module for AppServers

vFabric GemFire HTTP Session Management Module 7.0 for AppServers introduces the following integration enhancements:

• Integration of the module now only requires the addition of a single 'filter' element into the web application's web.xml file. You are no longer required to wrap every servlet and extend every JSP page from a special class.
• The modify_war script has been enhanced to ease integration of the module with EAR files. Using the script, it is now possible to build a self-contained EAR file which includes all necessary libraries to enable session replication.

For a list of application servers supported by this module, see vFabric Gemfire Modules Supported Configurations in the vFabric GemFire User's Guide for more information.

For more information on using the module, see HTTP Session Management Module for AppServers in the vFabric GemFire User's Guide for more information.

Improvements to Product Packaging and Distribution

Note the following packaging changes in vFabric GemFire 7.0:

• RPM distribution availability. You can now download and install vFabric GemFire from RPM on RedHat Enterprise Linux (RHEL). These RPMs are available on the product download page.
• ZIP file distribution. vFabric GemFire is no longer installable from a JAR installer. Product files are now distributed as a .zip file.
• Product code examples are bundled with product. Product examples (including the Tutorial and QuickStart files) are no longer a separate download. You can find the example files in the SampleCode directory under your product installation.
• VSD and DataBrowser tools are packaged with GemFire. These tools are available in the tools directory under your product installation.
• GemFire Pulse is included in the product. The new GemFire Pulse tool (which replaces GFMon) is packaged as a .WAR file and is bundled with vFabric GemFire. You can find the tool WAR file in the tools directory under your product installation.
• All GemFire HTTP Session Management Modules and the Hibernate Cache modules are now bundled with GemFire. The modules are no longer distributed as tar.gz files and are no longer a separate download. You can find the module installation .zip files in the tools/Modules directory under your product installation.

Changes to Product Documentation

vFabric GemFire documentation for 7.0 includes the following structural changes:

• Tool documentation is integrated into the vFabric GemFire User's Guide for more information. See Tools and Modules in the vFabric GemFire User's Guide for more information.
• The user's guide describes how Spring developers can use Spring Data GemFire to integrate vFabric GemFire components into their applications. See Getting Started with Spring Data GemFire in the vFabric GemFire User's Guide for more information.
• The vFabric GemFire User's Guide includes a topic that contains important GemFire-specific security information that helps you secure your environment.
• The vFabric GemFire Native Client Guide is now available in HTML format (as well as PDF). See the vFabric GemFire Native Client User's Guide.

Upgrading to vFabric GemFire 7.0

To upgrade from an earlier version of GemFire to the current version, see the Upgrading vFabric GemFire topic in the vFabric GemFire User's Guide under "Getting Started with vFabric GemFire." Also review product changes documented in What's New in vFabric GemFire 7.0.

In particular, if you are upgrading from vFabric GemFire 6.5 or 6.6 to 7.0, you will need to run the gfsh upgrade offline-disk-store command to upgrade your disk-stores to the 7.0 format. If you are upgrading from a version of GemFire earlier than 6.5 to 7.0, you need to run the DiskConverterPre65to65 tool on your disk-stores before running the upgrade offline-disk-store command.

In addition, if you are upgrading from a version of GemFire earlier than 6.6.0, see also the vFabric GemFire 6.6.1 and 6.6.0 Release Notes and vFabric GemFire 6.6.2 Release Notes to determine any additional changes required for you to migrate to vFabric GemFire 7.0.

Map of Old Commands to gfsh Commands in vFabric GemFire 7.0

The following table provides a mapping of previous GemFire commands (for example, gemfire, cacheserver, start locator, and so on) to new gfsh commands

Resolved Issues

For a list of bugs that are fixed in GemFire 7.0 and that are resolved in the VMware bug tracking system, see BugsFixedGemFire700.html.

Known Issues

For a list of issues that have been registered as bugs in the VMware bug tracking system, see BugNotesGemFire700.html.