Review Board 1.7.22


FLUME-1830: Flume User Guide missing some props for SpoolDirectorySource

Review Request #8941 - Created Jan. 14, 2013 and discarded

Alexander Alten-Lorenz
FLUME-1830
Reviewers
Flume
flume-git
Flume User Guide is missing these props (along with a description of what they do) that are present in the SpoolDirectorySource and LineDeserializer source file

 

Diff revision 1 (Latest)

  1. flume-ng-doc/sphinx/FlumeUserGuide.rst: Loading...
flume-ng-doc/sphinx/FlumeUserGuide.rst
Revision 63b8f9b New Change
[20] 790 lines
[+20]
791
             used, or files are modified while being read, the source will
791
             used, or files are modified while being read, the source will
792
             fail with an error message. For some use cases this may require
792
             fail with an error message. For some use cases this may require
793
             adding unique identifiers (such as a timestamp) to log file names
793
             adding unique identifiers (such as a timestamp) to log file names
794
             when they are copied into the spooling directory.
794
             when they are copied into the spooling directory.
795

    
   
795

   
796
====================  ==============  ==========================================================
796
==========================  ==============  ============================================================
797
Property Name         Default         Description
797
Property Name               Default         Description
798
====================  ==============  ==========================================================
798
==========================  ==============  ============================================================
799
**channels**          --
799
**channels**                --
800
**type**              --              The component type name, needs to be ``spooldir``
800
**type**                    --              The component type name, needs to be ``spooldir``
801
**spoolDir**          --              The directory where log files will be spooled
801
**spoolDir**                --              The directory where log files will be spooled
802
fileSuffix            .COMPLETED      Suffix to append to completely ingested files
802
fileSuffix                  .COMPLETED      Suffix to append to completely ingested files
803
fileHeader            false           Whether to add a header storing the filename
803
fileHeader                  false           Whether to add a header storing the filename
804
fileHeaderKey         file            Header key to use when appending filename to header
804
fileHeaderKey               file            Header key to use when appending filename to header
805
batchSize             10              Granularity at which to batch transfer to the channel
805
batchSize                   10              Granularity at which to batch transfer to the channel
806
bufferMaxLines        100             Maximum number of lines the commit buffer can hold
806
bufferMaxLines              100             Maximum number of lines the commit buffer can hold
807
bufferMaxLineLength   5000            Maximum length of a line in the commit buffer
807
bufferMaxLineLength         5000            Maximum length of a line in the commit buffer
808
selector.type         replicating     replicating or multiplexing
808
selector.type               replicating     replicating or multiplexing
809
selector.*                            Depends on the selector.type value
809
selector.*                                  Depends on the selector.type value
810
interceptors          --              Space separated list of interceptors
810
interceptors                --              Space separated list of interceptors
811
interceptors.*
811
interceptors.*
812
====================  ==============  ==========================================================
812
ignorePattern               ^$              Pattern of file to ignore (per default no effect)

    
   
813
metaDir                     .flumespool     Directory to store metadata about files being processed

    
   
814
deserializer                LINE            Deserializer to use to parse the file data into Flume Events

    
   
815
deserializer.outputCharset  UTF-8           Output charset

    
   
816
deserializer.maxLineLength  2048            Max length of a line to parse, line will be truncated after

    
   
817
==========================  ==============  ============================================================
813

    
   
818

   
814
Example for agent named a1:
819
Example for agent named a1:
815

    
   
820

   
816
.. code-block:: properties
821
.. code-block:: properties
817

    
   
822

   
[+20] [20] 1940 lines
  1. flume-ng-doc/sphinx/FlumeUserGuide.rst: Loading...