About
Logstash pipeline parser 0.0.2
![digraph G {
compound=true;
ratio="compress"
subgraph cluster_0 {
label = "plugin";
1 [label="input"];
1 -> 2;
subgraph cluster_7 {
label = "plugin type";
style = "dashed"
labelloc="bottom";
2 [label="syslog"];
3 [label="port"];
4 [label="5044"];
5 [label="codec"];
6 [label="cef"];
2 -> {3 5};
3 -> 4;
5 -> 6;
}
}
subgraph cluster_2 {
label = "plugin";
16 [label="filter"];
17 [label="if"];
24 [label="else if"];
subgraph cluster_6 {
label = "filter plugin";
style = "dashed";
labelloc="bottom";
21 [label="copy"];
22 [label="source"];
23 [label="target"];
21 -> 22;
22 -> 23;
}
subgraph cluster_8 {
label = "filter plugin";
style = "dashed";
labelloc="bottom";
33 [label="json"];
34 [label="message"];
35 [label="target"];
36 [label="source"];
37 [label="doc"];
33 -> {35 36};
36 -> 34;
35 -> 37;
}
subgraph cluster_9 {
label = "filter plugin";
style = "dashed";
labelloc="bottom";
38 [label="mutate"];
39 [label="rename"];
40 [label="shortHostname"];
41 [label="hostname"];
38 -> 39;
39 -> 40;
40 -> 41;
}
subgraph cluster_3 {
label = "condition";
labelloc="bottom";
style = "dashed";
20 [label="42" constraint=false];
19 [label=">=" constraint=false];
18 [label="[field]" constraint=false];
}
subgraph cluster_5 {
label = "filter plugin";
style = "dashed";
labelloc="bottom";
25 [label="mutate"];
26 [label="convert"];
27 [label="field"];
28 [label="integer"];
25 -> 26;
26 -> 27;
27 -> 28;
}
subgraph cluster_4 {
label = "condition";
labelloc="bottom"
style = "dashed";
labelloc="bottom";
32 [label="path2"];
31 [label="path1"];
30 [label="in"];
29 [label="[path]"];
}
16 -> {24 17};
30 -> 38 [ltail=cluster_4 minlen=2];
30 -> 33 [ltail=cluster_4 minlen=2];
30 -> 21 [ltail=cluster_4 minlen=2];
19 -> 25 [ltail=cluster_3 minlen=2];
17 -> {18 19 20};
24 -> {29 30 31 32};
}
subgraph cluster_1 {
label = "plugin";
7 [label="output"];
7 -> 8;
subgraph cluster_10 {
label = "plugin type";
labelloc="bottom";
style = "dashed";
labelloc="bottom";
8 [label="elasticsearch"];
9 [label="index"];
10 [label="my-index"];
13 [label="hosts"];
14 [label="es-host1"];
15 [label="es-host2"];
8 -> {9 13};
9 -> 10;
13 -> {14 15};
}
}
}](_images/graphviz-7b2bfb3c418bad5c2c2e9829641e35ec53b274bb.png)
What is Logstash pipeline
The Logstash event processing pipeline has three stages: inputs → filters → outputs. Inputs generate events, filters modify them, and outputs ship them elsewhere. Inputs and outputs support codecs that enable you to encode or decode the data as it enters or exits the pipeline without having to use a separate filter.
The pipeline configuration file is a custom format developed by the Logstash folks using Treetop. The grammar itself is described in the source file grammar.treetop and compiled using Treetop into the custom grammar.rb parser. That parser is then used to set up the pipeline from the Logstash configuration.
Dependencies
The parser has only one dependency on the pyparsing package for creating PEG parser
Issue tracking
You can find existing and fixed bugs by clicking on Issues and using “New Issue” to report previously unknown issues.