node-zwave-js/docs/getting-started/migrating/v14.md

Migrating to v14

This version became necessary more or less by accident. I originally planned to split some of the files that have grown too large (like Node.ts) into multiple smaller chunks. During this refactor, I noticed that several methods depended on a full-blown driver instance (or an appropriate abstraction), when all that was needed was access to the value DB for example. This problem appeared throughout the entire codebase, so I decided to do something about it. And since I was anyways going to touch almost everything, I opted to do a few more changes that required touching the whole codebase.

The ZWaveHost and ZWaveApplicationHost interfaces have been replaced by multiple smaller interfaces, each defining a specific subset of the functionality. All the code that previously expected to be passed one of those interfaces has been update to just accept what it actually needs. The Driver class still implements all of this functionality.

Furthermore, Message and CommandClass implementations are no longer bound to a specific host instance. Instead, their methods that need access to host functionality (like value DBs, home ID, device configuration, etc.) now receive a method-specific context object. Parsing of those instances no longer happens in the constructor, but in a separate from method.

Last but not least, the npm packages of Z-Wave JS are now hybrid ES Module & CommonJS. This means that consumers of Z-Wave JS can now benefit from the advantages of ES Modules, like tree-shaking or improved browser compatibility, without breaking compatibility with CommonJS-based projects.

All in all, this release contains a huge list of breaking changes, but most of those are limited to low-level or sparingly used APIs.

Replaced Node.js Buffer with Uint8Array or portable Bytes class

Since the beginning of Z-Wave JS, we've been using Node.js's Buffer class to manipulate binary data. This works fine, but is not portable, and prevents us from exploring compatibility with other runtimes, or even doing things in the browser, e.g. flashing controllers or modifying NVM contents. Following Sindre Sorhus's example, the use of Buffers was replaced with Uint8Arrays where applicable.

In input positions where Z-Wave JS previously accepted Buffers, this change is backwards compatible, as Buffer is a subclass of Uint8Array. Applications can simply continue passing Buffer instances to Z-Wave JS.

In output positions however, this is a breaking change. Where applications are affected by this, check if Buffer methods like readUInt32BE etc. are actually needed, or if changing the expected type to Uint8Array would be enough. This is usually the case when just passing the binary data around, or accessing its content by index.

In some cases, Z-Wave JS now uses an almost drop-in replacement for Node.js's Buffer, the new Bytes class exported from @zwave-js/shared. This is a portable subclass of Uint8Array with some additions to make its API more (but not 100%) compatible with Buffer. It supports most of the Buffer functionality like from(), concat(), toString() (with limitations), read/write[U]Int...LE/BE. This should generally not leak into the public API, except for some rare edge cases.

Both Uint8Array and Bytes can easily be converted to a Buffer instance if absolutely necessary by using Buffer.from(...).

To test whether something is a Uint8Array, use the isUint8Array function exported from node:util/types (not portable) or @zwave-js/shared (portable).

Configuration DB updates require an external config directory

Previously, when a new version of the config DB should be installed, Z-Wave JS would attempt to update @zwave-js/config inside node_modules, unless it was installed in Docker, or an external configuration directory was configured. This exception is however true for a majority of our installations, so the added complexity of dealing with package managers was not worth it.

Going forward, the Driver.installConfigUpdate() method will only install configuration updates if an external config directory is configured using either the deviceConfigExternalDir driver option or the ZWAVEJS_EXTERNAL_CONFIG environment variable.

Changed some default paths

A few default paths have been changed to be relative to the cwd of the current process, rather than Z-Wave JS's source files:

• The default logfile location
• The cache files of the driver
• The cache files of the flasher utility

If an application relies on their location, it should set the paths explicitly.

Decoupled Serial API messages from host instances, split constructors and parsing

Message instances no longer store a reference to their host. They are now "just data". Therefore, message constructors no longer take an instance of ZWaveHost as the first parameter. All needed information is passed to the relevant methods as context arguments.

Old: constructor for creation and parsing

public constructor(
	options:
		| MessageDeserializationOptions
		| MyMessageOptions,
) {
	super(options);

	if (gotDeserializationOptions(options)) {
		// deserialize message from this.payload
	} else {
		// populate message properties from options
	}
}

New: constructor for creation, separate method for parsing

public constructor(
	options: MyMessageOptions & MessageBaseOptions,
) {
	super(options);
	// populate message properties from options
}

public static from(
	raw: MessageRaw,
	ctx: MessageParsingContext,
): MyMessage {
	// deserialize message from raw.payload

	return new this({
		// ...
	});
}

[!NOTE]: For messages that contain a serialized CC instance, Message.parse no longer deserializes it automatically. This has to be done in a separate step.

Decoupled CCs from host instances, split constructors and parsing

CommandClass instances no longer store a reference to their host. They are now "just data". Therefore, CC constructors no longer take an instance of ZWaveHost as the first parameter. All needed information is passed to the relevant methods as context arguments.

Old: constructor for creation and parsing

public constructor(
	host: ZWaveHost,
	options: CommandClassDeserializationOptions | BinarySwitchCCSetOptions,
) {
	super(host, options);
	if (gotDeserializationOptions(options)) {
		validatePayload(this.payload.length >= 1);
		this.targetValue = !!this.payload[0];
		if (this.payload.length >= 2) {
			this.duration = Duration.parseSet(this.payload[1]);
		}
	} else {
		this.targetValue = options.targetValue;
		this.duration = Duration.from(options.duration);
	}
}

New: constructor for creation, separate method for parsing

public constructor(
	options: WithAddress<BinarySwitchCCSetOptions>,
) {
	super(options);
	this.targetValue = options.targetValue;
	this.duration = Duration.from(options.duration);
}

public static from(raw: CCRaw, ctx: CCParsingContext): BinarySwitchCCSet {
	validatePayload(raw.payload.length >= 1);
	const targetValue = !!raw.payload[0];

	let duration: Duration | undefined;
	if (raw.payload.length >= 2) {
		duration = Duration.parseSet(raw.payload[1]);
	}

	return new BinarySwitchCCSet({
		nodeId: ctx.sourceNodeId,
		targetValue,
		duration,
	});
}

Updated CC method signatures:

- interview(applHost: ZWaveApplicationHost): Promise<void>;
+ interview(ctx: InterviewContext): Promise<void>;

- refreshValues(applHost: ZWaveApplicationHost): Promise<void>;
+ refreshValues(ctx: RefreshValuesContext): Promise<void>;

- persistValues(applHost: ZWaveApplicationHost): Promise<void>;
+ persistValues(ctx: PersistValuesContext): Promise<void>;

- toLogEntry(host?: ZWaveValueHost): MessageOrCCLogEntry;
+ toLogEntry(ctx?: GetValueDB): MessageOrCCLogEntry;

[!NOTE]: Applications calling these methods can simply pass the driver instance.

Moved Serial API message implementations to @zwave-js/serial package

All serial API message implementations have been moved to the @zwave-js/serial package. In addition to the main entry point, they are also available via @zwave-js/serial/serialapi.

Other notable changes

These are the only ones I found to break dependent projects.

• All getNodeUnsafe methods have been renamed to tryGetNode to better indicate what they do - it was not really clear what was "unsafe" about them.
• The CommandClass.version property has been removed. To determine the CC version a node or endpoint supports, use the node.getCCVersion(cc) or endpoint.getCCVersion(cc) methods.
• The TXReport object no longer has the numRepeaters property, as this was implied by the length of the repeaterNodeIds array.

Other breaking changes

Most of these should not affect any application:

• The ZWaveHost and ZWaveApplicationHost interfaces have been split into multiple smaller "traits"
• The Driver class still implements all those traits, but receiving functions now just require the necessary subset.
• CC Constructors no longer take an instance of ZWaveHost as the first parameter. All needed information has been merged into the CCParsingContext type.
• The CommandClass.from method no longer takes an instance of ZWaveHost as the first parameter for the same reason.
• CommandClass instances no longer store a reference to their host. They are now "just data".
• The serialize() implementations of CCs now take an argument of type CCEncodingContext
• The toLogEntry() implementations of CCs now take an argument of type GetValueDB instead of ZWaveHost
• The interview() implementations of CCs now take an argument of type InterviewContext instead of ZWaveApplicationHost
• The refreshValues() implementations of CCs now take an argument of type RefreshValuesContext instead of ZWaveApplicationHost
• The persistValues() implementations of CCs now take an argument of type PersistValuesContext instead of ZWaveApplicationHost
• The translateProperty() and translatePropertyKey() implementations of CCs now take an argument of type GetValueDB instead of ZWaveApplicationHost
• The mergePartialCCs signature has changed:

  -    mergePartialCCs(applHost: ZWaveApplicationHost, partials: CommandClass[]): void;
  +    mergePartialCCs(partials: CommandClass[], ctx: CCParsingContext): void;
  

• The ICommandClass interface has been removed. Where applicable, it has been replaced with the CCId interface whose sole purpose is to identify the command and destination of a CC.
• The IZWaveNode, IZWaveEndpoint, IVirtualNode and IVirtualEndpoint interfaces have been replaced with more specific "trait" interfaces.
• The TestingHost, TestNode and TestEndpoint interfaces and implementations have been reworked to be more declarative and require less input in tests.
• Serial API Message constructors no longer take an instance of ZWaveHost as the first parameter. All needed information has been merged into the MessageParsingContext type.
• The Message.from method no longer takes an instance of ZWaveHost as the first parameter for the same reason.
• The callbackId of a Message instance is no longer automatically generated when reading it for the first time. Instead, the callbackId can be undefined and has to be explicitly set before serializing if necessary.
• The serialize() implementations of Messages now take an argument of type MessageEncodingContext
• The Message.getNodeUnsafe() method has been renamed to Message.tryGetNode
• The Driver.getNodeUnsafe(message) method has been renamed to Driver.tryGetNode(message)
• Driver.controllerLog is no longer public.
• Driver.getSafeCCVersion can now return undefined if the requested CC is not implemented in zwave-js
• Driver.isControllerNode(nodeId) has been removed. Compare with ownNodeId instead or use the Node.isControllerNode property.
• The Endpoint.getNodeUnsafe() method has been renamed to tryGetNode
• The [CCName]CC[CCCommand]Options interfaces no longer extend another interface and now exclusively contain the CC-specific properties required for constructing that particular CC
• CC-specific constructors now accept a single options object of type WithAddress<...>, which is the CC-specific options interface, extended by nodeId and endpointIndex. Note that endpointIndex was previously just called endpoint.
• The CommandClassDeserializationOptions interface and the gotDeserializationOptions method no longer exist. Instead, CCs parse their specific payload using the new static from method, which takes a pre-parsed CCRaw (ccId, ccCommand, binary payload) and the CCParsingContext. All CCs with a custom constructor implementations are expected to implement this aswell. To parse an arbitrary CC buffer, use CommandClass.parse(buffer, context).
• Several CCs had their constructor options type reworked slightly for correctness. In some cases it is now necessary to pass properties that were previously inferred.
• The CommandClassCreationOptions type has been merged into the CommandClassOptions type, which now consists only of the CC destination and optional overrides for ccId, ccCommand and payload.
• CommandClass.deserialize has been removed. It's functionality is now provided by the parse method and CCRaw.
• CommandClass.getCommandClass, CommandClass.getCCCommand have been removed. Their functionality is now provided by CCRaw.parse
• CommandClass.getConstructor has been removed. If really needed, the functionality is available via the getCCConstructor and getCCCommandConstructor methods exported by @zwave-js/cc.
• The origin property of the CommandClass class was removed, since it served no real purpose.
• The [MessageName]Options interfaces no longer extend another interface and now exclusively contain the message-specific properties required for constructing that particular instance
• Message-specific constructors now accept a single options object of type [MessageName]Options & MessageBaseOptions.
• The MessageDeserializationOptions interface and the gotDeserializationOptions method no longer exist
• The DeserializingMessageConstructor interface no longer exists
• Instead, message instances parse their specific payload using the new static from method, which takes a pre-parsed MessageRaw (message type, function type, binary payload) and the MessageParsingContext. All Message implementations with a custom constructor are expected to implement the from method aswell. To parse an arbitrary message buffer use Message.parse(buffer, context).
• The MessageCreationOptions type has been renamed to MessageOptions
• Several Message implementations had their constructor options type reworked slightly for correctness. In some cases it is now necessary to pass properties that were previously inferred.
• The static Message methods extractPayload, getConstructor, getMessageLength and isComplete were removed.
• The Message.options property was removed
• The ICommandClassContainer interface was replaced with the new ContainsCC interface, which indicates that something contains a deserialized CC instance.
• The Driver.computeNetCCPayloadSize method now requires that the passed message instance contains a deserialized CC instance.
• The INodeQuery interface and the isNodeQuery function were removed. To test whether a message references a node, use hasNodeId(msg) instead, which communicates the intent more clearly.