Review Board 1.7.22


FLUME-2238 - Provide option to configure worker threads in NettyAvroRpcClient

Review Request #15683 - Created Nov. 19, 2013 and updated

Cameron Gandevia
FLUME-2238
Reviewers
Flume
flume-git
Provide option to configure worker threads in NettyAvroRpcClient

 
flume-ng-doc/sphinx/FlumeUserGuide.rst
Revision 0f12427 New Change
[20] 1634 lines
[+20]
1635
sent to this sink are turned into Avro events and sent to the configured
1635
sent to this sink are turned into Avro events and sent to the configured
1636
hostname / port pair. The events are taken from the configured Channel in
1636
hostname / port pair. The events are taken from the configured Channel in
1637
batches of the configured batch size.
1637
batches of the configured batch size.
1638
Required properties are in **bold**.
1638
Required properties are in **bold**.
1639

    
   
1639

   
1640
==========================   =======  ==============================================
1640
==========================   =====================================================  ===========================================================================================
1641
Property Name                Default  Description
1641
Property Name                Default  Description
1642
==========================   =======  ==============================================
1642
==========================   =====================================================  ===========================================================================================
1643
**channel**                  --
1643
**channel**                  --
1644
**type**                     --       The component type name, needs to be ``avro``.
1644
**type**                     --                                                     The component type name, needs to be ``avro``.
1645
**hostname**                 --       The hostname or IP address to bind to.
1645
**hostname**                 --                                                     The hostname or IP address to bind to.
1646
**port**                     --       The port # to listen on.
1646
**port**                     --                                                     The port # to listen on.
1647
batch-size                   100      number of event to batch together for send.
1647
batch-size                   100                                                    number of event to batch together for send.
1648
connect-timeout              20000    Amount of time (ms) to allow for the first (handshake) request.
1648
connect-timeout              20000                                                  Amount of time (ms) to allow for the first (handshake) request.
1649
request-timeout              20000    Amount of time (ms) to allow for requests after the first.
1649
request-timeout              20000                                                  Amount of time (ms) to allow for requests after the first.
1650
reset-connection-interval    none     Amount of time (s) before the connection to the next hop is reset. This will force the Avro Sink to reconnect to the next hop. This will allow the sink to connect to hosts behind a hardware load-balancer when news hosts are added without having to restart the agent.
1650
reset-connection-interval    none                                                   Amount of time (s) before the connection to the next hop is reset. This will force the Avro Sink to reconnect to the next hop. This will allow the sink to connect to hosts behind a hardware load-balancer when news hosts are added without having to restart the agent.
1651
compression-type             none     This can be "none" or "deflate".  The compression-type must match the compression-type of matching AvroSource
1651
compression-type             none                                                   This can be "none" or "deflate".  The compression-type must match the compression-type of matching AvroSource
1652
compression-level            6        The level of compression to compress event. 0 = no compression and 1-9 is compression.  The higher the number the more compression
1652
compression-level            6                                                      The level of compression to compress event. 0 = no compression and 1-9 is compression.  The higher the number the more compression
1653
ssl                          false    Set to true to enable SSL for this AvroSink. When configuring SSL, you can optionally set a "truststore", "truststore-password", "truststore-type", and specify whether to "trust-all-certs".
1653
ssl                          false                                                  Set to true to enable SSL for this AvroSink. When configuring SSL, you can optionally set a "truststore", "truststore-password", "truststore-type", and specify whether to "trust-all-certs".
1654
trust-all-certs              false    If this is set to true, SSL server certificates for remote servers (Avro Sources) will not be checked. This should NOT be used in production because it makes it easier for an attacker to execute a man-in-the-middle attack and "listen in" on the encrypted connection.
1654
trust-all-certs              false                                                  If this is set to true, SSL server certificates for remote servers (Avro Sources) will not be checked. This should NOT be used in production because it makes it easier for an attacker to execute a man-in-the-middle attack and "listen in" on the encrypted connection.
1655
truststore                   --       The path to a custom Java truststore file. Flume uses the certificate authority information in this file to determine whether the remote Avro Source's SSL authentication credentials should be trusted. If not specified, the default Java JSSE certificate authority files (typically "jssecacerts" or "cacerts" in the Oracle JRE) will be used.
1655
truststore                   --                                                     The path to a custom Java truststore file. Flume uses the certificate authority information in this file to determine whether the remote Avro Source's SSL authentication credentials should be trusted. If not specified, the default Java JSSE certificate authority files (typically "jssecacerts" or "cacerts" in the Oracle JRE) will be used.
1656
truststore-password          --       The password for the specified truststore.
1656
truststore-password          --                                                     The password for the specified truststore.
1657
truststore-type              JKS      The type of the Java truststore. This can be "JKS" or other supported Java truststore type.
1657
truststore-type              JKS                                                    The type of the Java truststore. This can be "JKS" or other supported Java truststore type.
1658
==========================   =======  ==============================================
1658
maxIoWorkers                 2 * the number of available processors in the machine  The maximum number of I/O worker threads. This is configured on the NettyAvroRpcClient NioClientSocketChannelFactory.

    
   
1659
==========================   =====================================================  ===========================================================================================
1659

    
   
1660

   
1660
Example for agent named a1:
1661
Example for agent named a1:
1661

    
   
1662

   
1662
.. code-block:: properties
1663
.. code-block:: properties
1663

    
   
1664

   
[+20] [20] 1650 lines
flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
Revision 9aabdd4 New Change
 
flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java
Revision 7aa70cb New Change
 
flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
Revision bfb1fa6 New Change
 
  1. flume-ng-doc/sphinx/FlumeUserGuide.rst: Loading...
  2. flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java: Loading...
  3. flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java: Loading...
  4. flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java: Loading...