Pipeline

class logstash_pipeline_parser.Pipeline(pipeline: str)

Parameters:

pipeline (str) – The pipeline definition

A class representing the Logstash pipeline.

add_type(name: str, new_type: type[Any] | Callable[[Any], Any]) → Self

Parameters:

• name (str) – Type name

• new_type (type[Any] | Callable[[Any], Any]) – New type for given name.

Return type:

Pipeline

Adds a new type.

For example function:

from logstash_parser import Pipeline

Pipeline("").add_type("port", str)

For example class:

from logstash_parser import Pipeline

class MyPortType:
    pass

Pipeline("").add_type("port", MyPortType)

Note

Please see Types for more examples.

classmethod from_file(path: str | Path) → Self

Parameters:

path (str | pathlib.Path) – Path to the file

Return type:

Pipeline

Instantiates class from a file.

For example from string:

from logstash_parser import Pipeline

pipeline = Pipeline.from_file("/some/path/to/pipeline.conf")

Or from Path:

from logstash_parser import Pipeline
from pathlib import Path

path = Path("/some/path/to/pipeline.conf")
pipeline = Pipeline.from_file(path)

get_types() → dict[str, type[Any] | Callable[[Any], Any]]

Returns:

All names as key and types as value.

Return type:

dict

Returns all defined types

Predefined types are:

Name	Type/Callable
aggregate_maps_path	pathlib.Path
cacert	pathlib.Path
ca_file	pathlib.Path
client_cert	pathlib.Path
client_key	pathlib.Path
database	pathlib.Path
dictionary_path	pathlib.Path
jaas_path	pathlib.Path
jdbc_driver_library	pathlib.Path
jdbc_password_filepath	pathlib.Path
json_key_file	pathlib.Path
kerberos_config	pathlib.Path
keystore	pathlib.Path
mib_paths	pathlib.Path
network_path	pathlib.Path
path	pathlib.Path
private_key	pathlib.Path
processed_db_path	pathlib.Path
public_key	pathlib.Path
schema_registry_ssl_keystore_location	pathlib.Path
schema_registry_ssl_truststore_location	pathlib.Path
send_nsca_config	pathlib.Path
ssl_cacert	pathlib.Path
ssl_cert	pathlib.Path
ssl_certificate	pathlib.Path
ssl_certificate_path	pathlib.Path
ssl_key	pathlib.Path
ssl_keystore_location	pathlib.Path
ssl_keystore_path	pathlib.Path
ssl_truststore_location	pathlib.Path
ssl_truststore_path	pathlib.Path
statement_filepath	pathlib.Path
template	pathlib.Path
template_file	pathlib.Path
truststore	pathlib.Path

parse() → list

Returns:

Parsed tree

Return type:

list

Create an Abstract syntax tree from the input data. Of course, it is possible to parse all kinds of plugins, conditions and data types.

For example this input:

from logstash_pipeline_parser import Pipeline

data = r"""
input {
  beats {
    host => "0.0.0.0"
    port => 5044
    client_inactivity_timeout => 3600
    include_codec_tag => true
    enrich => [source_metadata, ssl_peer_metadata]
    ssl => true
    ssl_key => "/some/path/my.key"
    id => "input_beats"
  }
}
"""

ast = Pipeline(data).parse()

will produce this array:

from ipaddress import IPv4Address
from pathlib import Path

[
    ["input",[
        ["beats", [
            ["host", [IPv4Address("0.0.0.0")]],
            ["port", [5044]],
            ["client_inactivity_timeout", [3600]],
            ["include_codec_tag", [True]],
            ["enrich", [
                ["source_metadata", "ssl_peer_metadata"]
            ]],
            ["ssl", [True]],
            ["ssl_key", [Path("/some/path/my.key")]],
            ["id", ["input_beats"]]
        ]]
    ]]
]

remove_type(name: str) → Self

Parameters:

name (str) – Type name

Return type:

Pipeline

Removes a type

For example function:

from logstash_parser import Pipeline

Pipeline("").remove_type("port")

search(key: str) → Generator[tuple[str, Any], None, None]

Parameters:

key (str) – Key name to search for

Returns:

Found values in the form tuple[key, value]

Return type:

collections.abc.Generator[tuple[str, Any], None, None]

Yield the searched keys and their values from the tree. The key can also contain the wildcard *, for example “output.*.hosts” will return (if the pipeline definition contains them):

- ("output.elasticsearch.hosts", ["127.0.0.1:9200","127.0.0.2:9200"])
- ("output.logstash.hosts", "127.0.0.1:9801")

Note

Please see Search for more examples.