Package org.nuxeo.lib.stream.log

Interface LogManager

All Superinterfaces:

AutoCloseable

All Known Implementing Classes:

AbstractLogManager, ChronicleLogManager, KafkaLogManager, UnifiedLogManager

public interface LogManager
extends AutoCloseable

Manage Log and give access to Appenders and Tailers. Closing the LogManager will also close all its appenders and tailers.

Since:

9.3

Method Summary

Modifier and Type	Method	Description
void	close()
default boolean	createIfNotExists(String name, int size)	Deprecated. since 11.1 use createIfNotExists(Name, int) instead
boolean	createIfNotExists(Name name, int size)	Creates a new Log with size partitions if the Log does not exists.
default <M extends Externalizable> LogTailer<M>	createTailer(String group, String name)	Deprecated. since 11.1 use createTailer(Name, Name) (Name)} instead
default <M extends Externalizable> LogTailer<M>	createTailer(String group, String name, Codec<M> codec)	Deprecated. since 11.1 use createTailer(Name, Name, Codec) instead
default <M extends Externalizable> LogTailer<M>	createTailer(String group, Collection<LogPartition> partitions)	Deprecated. since 11.1 use (Name) instead
default <M extends Externalizable> LogTailer<M>	createTailer(String group, Collection<LogPartition> partitions, Codec<M> codec)	Deprecated. since 11.1 use createTailer(Name, Name, Codec) (Name)} instead
default <M extends Externalizable> LogTailer<M>	createTailer(String group, LogPartition partition)	Deprecated. since 11.1 use (Name) instead
default <M extends Externalizable> LogTailer<M>	createTailer(String group, LogPartition partition, Codec<M> codec)	Deprecated. since 11.1 use createTailer(Name, LogPartition, Codec) instead
default <M extends Externalizable> LogTailer<M>	createTailer(Name group, Collection<LogPartition> partitions)	Creates a tailer for a consumer group and assign multiple partitions.
<M extends Externalizable> LogTailer<M>	createTailer(Name group, Collection<LogPartition> partitions, Codec<M> codec)	Creates a tailer for a consumer group and assign multiple partitions.
default <M extends Externalizable> LogTailer<M>	createTailer(Name group, LogPartition partition)	Creates a tailer for a consumer group and assign a single partition.
default <M extends Externalizable> LogTailer<M>	createTailer(Name group, LogPartition partition, Codec<M> codec)	Creates a tailer for a consumer group and assign a single partition.
default <M extends Externalizable> LogTailer<M>	createTailer(Name group, Name name)	Creates a tailer for a consumer group and assign all partitions of the Log.
default <M extends Externalizable> LogTailer<M>	createTailer(Name group, Name name, Codec<M> codec)	Creates a tailer for a consumer group and assigns all partitions of the Log.
default boolean	delete(String name)	Deprecated. since 11.1 use delete(Name) instead
boolean	delete(Name name)	Tries to delete a Log.
default void	deleteConsumers()	Remove all existing consumers and their committed positions.
default void	deleteRecords(Name name)	Delete all records of a stream by moving the first offsets to end of each partition.
default boolean	exists(String name)	Deprecated. since 11.1 use exists(Name) instead
boolean	exists(Name name)	Returns true if a Log with this name exists.
default <M extends Externalizable> LogAppender<M>	getAppender(String name)	Deprecated. since 11.1 use getAppender(Name) instead
default <M extends Externalizable> LogAppender<M>	getAppender(String name, Codec<M> codec)	Deprecated. since 11.1 use getAppender(Name, Codec) instead
default <M extends Externalizable> LogAppender<M>	getAppender(Name name)	Gets an appender for the Log named name, uses an already defined codec or the legacy encoding to encode records.
<M extends Externalizable> LogAppender<M>	getAppender(Name name, Codec<M> codec)	Gets an appender for the Log named name, uses codec to encode records.
default LogLag	getLag(String name, String group)	Deprecated. since 11.1 use getLag(Name, Name) instead
default LogLag	getLag(Name name, Name group)	Returns the lag between consumer group and producers for a Log.
default List<LogLag>	getLagPerPartition(String name, String group)	Deprecated. since 11.1 use getLagPerPartition(Name, Name) instead
List<LogLag>	getLagPerPartition(Name name, Name group)	Returns the lag between consumer group and the producers for each partition.
default <M extends Externalizable> Latency	getLatency(String name, String group, Codec<M> codec, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor)	Deprecated. since 11.1 use getLatencyPerPartition(Name, Name, Codec, Function, Function) instead
default <M extends Externalizable> Latency	getLatency(Name name, Name group, Codec<M> codec, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor)	Returns the latency between consumer group and producers for a Log.
default <M extends Externalizable> List<Latency>	getLatencyPerPartition(String name, String group, Codec<M> codec, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor)	Deprecated. since 11.1 use getLatencyPerPartition(Name, Name, Codec, Function, Function) instead
<M extends Externalizable> List<Latency>	getLatencyPerPartition(Name name, Name group, Codec<M> codec, Function<M,Long> timestampExtractor, Function<M,String> keyExtractor)	Returns the lag with latency.
default List<String>	listAll()	Deprecated. since 11.1 use listAllNames() instead
List<Name>	listAllNames()	Returns all the Log names.
default List<String>	listConsumerGroups(String name)	Deprecated. since 11.1 use listConsumerGroups(Name) instead
List<Name>	listConsumerGroups(Name name)	List the consumer groups for a Log.
default int	size(String name)	Deprecated. since 11.1 use size(Name) instead
int	size(Name name)	Returns the number of partition of a Log.
default <M extends Externalizable> LogTailer<M>	subscribe(String group, Collection<String> names, RebalanceListener listener)	Deprecated. since 11.1 use subscribe(Name, Collection, RebalanceListener) instead
default <M extends Externalizable> LogTailer<M>	subscribe(String group, Collection<String> names, RebalanceListener listener, Codec<M> codec)	Deprecated. since 11.1 use subscribe(Name, Collection, RebalanceListener, Codec) instead
default <M extends Externalizable> LogTailer<M>	subscribe(Name group, Collection<Name> names, RebalanceListener listener)
<M extends Externalizable> LogTailer<M>	subscribe(Name group, Collection<Name> names, RebalanceListener listener, Codec<M> codec)	Creates a tailer for a consumer group and subscribe to multiple Logs.
boolean	supportSubscribe()	Deprecated. since 2021.34 use supportSubscribe(Name) instead
boolean	supportSubscribe(Name stream)	Returns true if the subscribe(org.nuxeo.lib.stream.log.Name, java.util.Collection<org.nuxeo.lib.stream.log.Name>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported for the specific Log.

Method Detail

exists

boolean exists(Name name)

Returns true if a Log with this name exists.

Since:

11.1

exists

@Deprecated(since="11.1")
default boolean exists(String name)

Deprecated.

since 11.1 use exists(Name) instead

createIfNotExists

boolean createIfNotExists(Name name,
                          int size)

Creates a new Log with size partitions if the Log does not exists. Returns true it the Log has been created.

Since:

11.1

createIfNotExists

@Deprecated(since="11.1")
default boolean createIfNotExists(String name,
                                  int size)

Deprecated.

since 11.1 use createIfNotExists(Name, int) instead

delete

boolean delete(Name name)

Tries to delete a Log. Returns true if successfully deleted, might not be possible depending on the implementation.

Since:

11.1

delete

@Deprecated(since="11.1")
default boolean delete(String name)

Deprecated.

since 11.1 use delete(Name) instead

size

int size(Name name)

Returns the number of partition of a Log.

Since:

11.1

size

@Deprecated(since="11.1")
default int size(String name)

Deprecated.

since 11.1 use size(Name) instead

Returns the number of partition of a Log.

Since:

10.2

getAppender

<M extends Externalizable> LogAppender<M> getAppender(Name name,
                                                      Codec<M> codec)

Gets an appender for the Log named name, uses codec to encode records. An appender is thread safe.

Since:

10.2

getAppender

@Deprecated(since="11.1")
default <M extends Externalizable> LogAppender<M> getAppender(String name,
                                                              Codec<M> codec)

Deprecated.

since 11.1 use getAppender(Name, Codec) instead

getAppender

default <M extends Externalizable> LogAppender<M> getAppender(Name name)

Gets an appender for the Log named name, uses an already defined codec or the legacy encoding to encode records. An appender is thread safe.

Since:

11.1

getAppender

@Deprecated(since="11.1")
default <M extends Externalizable> LogAppender<M> getAppender(String name)

Deprecated.

since 11.1 use getAppender(Name) instead

Gets an appender for the Log named name, uses an already defined codec or the legacy encoding to encode records. An appender is thread safe.

Since:

10.2

createTailer

<M extends Externalizable> LogTailer<M> createTailer(Name group,
                                                     Collection<LogPartition> partitions,
                                                     Codec<M> codec)

Creates a tailer for a consumer group and assign multiple partitions. Uses codec to decode records. Note that partitions can be from different Logs. A tailer is NOT thread safe.

Since:

11.1

createTailer

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             Collection<LogPartition> partitions,
                                                             Codec<M> codec)

Deprecated.

since 11.1 use createTailer(Name, Name, Codec) (Name)} instead

Creates a tailer for a consumer group and assign multiple partitions. Uses codec to decode records. Note that partitions can be from different Logs. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(Name group,
                                                             Collection<LogPartition> partitions)

Creates a tailer for a consumer group and assign multiple partitions. Note that partitions can be from different Logs. Reads records using the legacy decoder. A tailer is NOT thread safe.

Since:

11.1

createTailer

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             Collection<LogPartition> partitions)

Deprecated.

since 11.1 use (Name) instead

Creates a tailer for a consumer group and assign multiple partitions. Note that partitions can be from different Logs. Reads records using the legacy decoder. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(Name group,
                                                             LogPartition partition)

Creates a tailer for a consumer group and assign a single partition. Reads records using the legacy decoder. A tailer is NOT thread safe.

Since:

11.1

createTailer

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             LogPartition partition)

Deprecated.

since 11.1 use (Name) instead

Creates a tailer for a consumer group and assign a single partition. Reads records using the legacy decoder. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(Name group,
                                                             Name name)

Creates a tailer for a consumer group and assign all partitions of the Log. Reads records using the legacy decoder. A tailer is NOT thread safe.

Since:

11.1

createTailer

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             String name)

Deprecated.

since 11.1 use createTailer(Name, Name) (Name)} instead

Creates a tailer for a consumer group and assign all partitions of the Log. Reads records using the legacy decoder. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(Name group,
                                                             LogPartition partition,
                                                             Codec<M> codec)

Creates a tailer for a consumer group and assign a single partition. Use an explicit codec to decode records. A tailer is NOT thread safe.

Since:

11.1

createTailer

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             LogPartition partition,
                                                             Codec<M> codec)

Deprecated.

since 11.1 use createTailer(Name, LogPartition, Codec) instead

Creates a tailer for a consumer group and assign a single partition. Use an explicit codec to decode records. A tailer is NOT thread safe.

Since:

10.2

createTailer

default <M extends Externalizable> LogTailer<M> createTailer(Name group,
                                                             Name name,
                                                             Codec<M> codec)

Creates a tailer for a consumer group and assigns all partitions of the Log. Uses codec to decode records. A tailer is NOT thread safe.

Since:

11.1

createTailer

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> createTailer(String group,
                                                             String name,
                                                             Codec<M> codec)

Deprecated.

since 11.1 use createTailer(Name, Name, Codec) instead

Creates a tailer for a consumer group and assigns all partitions of the Log. Uses codec to decode records. A tailer is NOT thread safe.

Since:

10.2

supportSubscribe

boolean supportSubscribe()

Deprecated.

since 2021.34 use supportSubscribe(Name) instead

Returns true if the Log subscribe(org.nuxeo.lib.stream.log.Name, java.util.Collection<org.nuxeo.lib.stream.log.Name>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported. Now deprecated because some implementations support subscribe only on specific Log names.

supportSubscribe

boolean supportSubscribe(Name stream)

Returns true if the subscribe(org.nuxeo.lib.stream.log.Name, java.util.Collection<org.nuxeo.lib.stream.log.Name>, org.nuxeo.lib.stream.log.RebalanceListener, org.nuxeo.lib.stream.codec.Codec<M>) method is supported for the specific Log.

Since:

2021.34

subscribe

<M extends Externalizable> LogTailer<M> subscribe(Name group,
                                                  Collection<Name> names,
                                                  RebalanceListener listener,
                                                  Codec<M> codec)

Creates a tailer for a consumer group and subscribe to multiple Logs. The partitions assignment is done dynamically depending on the number of subscribers. The partitions can change during tailers life, this is called a rebalancing. A listener can be used to be notified on assignment changes. Uses codec to decode records.

Since:

11.1

Implementation Requirements:

You should not mix createTailer(org.nuxeo.lib.stream.log.Name, java.util.Collection<org.nuxeo.lib.stream.log.LogPartition>, org.nuxeo.lib.stream.codec.Codec<M>) and subscribe usage using the same group.

Implementation Note:

A tailer is NOT thread safe.

subscribe

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> subscribe(String group,
                                                          Collection<String> names,
                                                          RebalanceListener listener,
                                                          Codec<M> codec)

Deprecated.

since 11.1 use subscribe(Name, Collection, RebalanceListener, Codec) instead

Creates a tailer for a consumer group and subscribe to multiple Logs. The partitions assignment is done dynamically depending on the number of subscribers. The partitions can change during tailers life, this is called a rebalancing. A listener can be used to be notified on assignment changes. Uses codec to decode records.

Since:

10.2

Implementation Requirements:

You should not mix createTailer(org.nuxeo.lib.stream.log.Name, java.util.Collection<org.nuxeo.lib.stream.log.LogPartition>, org.nuxeo.lib.stream.codec.Codec<M>) and subscribe usage using the same group.

Implementation Note:

A tailer is NOT thread safe.

subscribe

default <M extends Externalizable> LogTailer<M> subscribe(Name group,
                                                          Collection<Name> names,
                                                          RebalanceListener listener)

subscribe

@Deprecated(since="11.1")
default <M extends Externalizable> LogTailer<M> subscribe(String group,
                                                          Collection<String> names,
                                                          RebalanceListener listener)

Deprecated.

since 11.1 use subscribe(Name, Collection, RebalanceListener) instead

Since:

10.2

getLagPerPartition

List<LogLag> getLagPerPartition(Name name,
                                Name group)

Returns the lag between consumer group and the producers for each partition. The result list is ordered, for instance index 0 is lag for partition 0.

Since:

11.1

getLagPerPartition

@Deprecated(since="11.1")
default List<LogLag> getLagPerPartition(String name,
                                        String group)

Deprecated.

since 11.1 use getLagPerPartition(Name, Name) instead

Returns the lag between consumer group and the producers for each partition. The result list is ordered, for instance index 0 is lag for partition 0.

Since:

10.2

getLag

default LogLag getLag(Name name,
                      Name group)

Returns the lag between consumer group and producers for a Log.

Since:

11.1

getLag

@Deprecated(since="11.1")
default LogLag getLag(String name,
                      String group)

Deprecated.

since 11.1 use getLag(Name, Name) instead

Returns the lag between consumer group and producers for a Log.

Since:

10.2

getLatencyPerPartition

<M extends Externalizable> List<Latency> getLatencyPerPartition(Name name,
                                                                Name group,
                                                                Codec<M> codec,
                                                                Function<M,Long> timestampExtractor,
                                                                Function<M,String> keyExtractor)

Returns the lag with latency. Timestamps used to compute the latencies are extracted from the records. This requires to read one record per partition so it costs more than getLagPerPartition(Name, Name).

Two functions need to be provided to extract the timestamp and a key from a record.

Since:

11.1

getLatencyPerPartition

@Deprecated(since="11.1")
default <M extends Externalizable> List<Latency> getLatencyPerPartition(String name,
                                                                        String group,
                                                                        Codec<M> codec,
                                                                        Function<M,Long> timestampExtractor,
                                                                        Function<M,String> keyExtractor)

Deprecated.

since 11.1 use getLatencyPerPartition(Name, Name, Codec, Function, Function) instead

Returns the lag with latency. Timestamps used to compute the latencies are extracted from the records. This requires to read one record per partition so it costs more than getLagPerPartition(Name, Name).

Two functions need to be provided to extract the timestamp and a key from a record.

Since:

10.2

getLatency

default <M extends Externalizable> Latency getLatency(Name name,
                                                      Name group,
                                                      Codec<M> codec,
                                                      Function<M,Long> timestampExtractor,
                                                      Function<M,String> keyExtractor)

Returns the latency between consumer group and producers for a Log.

Since:

11.1

getLatency

@Deprecated(since="11.1")
default <M extends Externalizable> Latency getLatency(String name,
                                                      String group,
                                                      Codec<M> codec,
                                                      Function<M,Long> timestampExtractor,
                                                      Function<M,String> keyExtractor)

Deprecated.

since 11.1 use getLatencyPerPartition(Name, Name, Codec, Function, Function) instead

Returns the latency between consumer group and producers for a Log.

Since:

10.2

listAll

@Deprecated(since="11.1")
default List<String> listAll()

Deprecated.

since 11.1 use listAllNames() instead

Returns all the Log names.

listAllNames

List<Name> listAllNames()

Returns all the Log names.

Since:

11.1

listConsumerGroups

List<Name> listConsumerGroups(Name name)

List the consumer groups for a Log.

Since:

11.1

Implementation Note:

Note that for Kafka it returns only consumers that use the subscribe API.

listConsumerGroups

@Deprecated(since="11.1")
default List<String> listConsumerGroups(String name)

Deprecated.

since 11.1 use listConsumerGroups(Name) instead

List the consumer groups for a Log.

Since:

10.2

Implementation Note:

Note that for Kafka it returns only consumers that use the subscribe API.

close

void close()

Specified by:

close in interface AutoCloseable

deleteRecords

default void deleteRecords(Name name)

Delete all records of a stream by moving the first offsets to end of each partition. Exposed for testing purpose, might not be implemented on all implementation.

Since:

2021.43

deleteConsumers

default void deleteConsumers()

Remove all existing consumers and their committed positions. Exposed for testing purpose, might not be implemented by all implementation.

Since:

2021.43