Well,不要刷屏了

logstash7.9.3 想要增量加载sqlserver 数据 出现报错:Error: invalid byte sequence in UTF-8

Logstash | 作者 jiping | 发布于2021年02月20日 | 阅读数:1439

[2021-02-20T16:03:09,095][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.3", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc Java HotSpot(TM) 64-Bit Server VM 25.31-b07 on 1.8.0_31-b13 +indy +jit [mswin32-x86_64]"}
[2021-02-20T16:03:09,389][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-02-20T16:03:11,820][INFO ][org.reflections.Reflections] Reflections took 43 ms to scan 1 urls, producing 22 keys and 45 values 
[2021-02-20T16:03:12,454][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/java/ELK/logstash-7.9.3/sqlserver/jdbc.conf"], :thread=>"#<Thread:0x17282b0e run>"}
[2021-02-20T16:03:13,251][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.77}
[2021-02-20T16:03:13,429][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-02-20T16:03:13,501][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2021-02-20T16:03:13,823][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-02-20T16:03:14,849][INFO ][logstash.inputs.jdbc     ][main][02673f0387aae5f6ebe5802e69b699cfbbb05452e0c21b78b224336e24314329] (0.119986s) SELECT CAST(SERVERPROPERTY('ProductVersion') AS varchar)
[2021-02-20T16:03:15,027][ERROR][logstash.javapipeline    ][main][02673f0387aae5f6ebe5802e69b699cfbbb05452e0c21b78b224336e24314329] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"sa", charset=>"UTF-8", jdbc_paging_enabled=>true, jdbc_page_size=>50000, statement_filepath=>"C:\\java\\ELK\\logstash-7.9.3\\sqlserver\\jdbc.sql", tracking_column_type=>"numeric", type=>"advancelist", record_last_run=>true, use_column_value=>true, tracking_column=>"ID", jdbc_password=><password>, clean_run=>false, jdbc_default_timezone=>"Asia/Shanghai", jdbc_driver_library=>"C:\\java\\ELK\\logstash-7.9.3\\sqlserver\\sqljdbc4.jar", jdbc_connection_string=>"jdbc:sqlserver://127.0.0.1:1433;DatabaseName=PKI", id=>"02673f0387aae5f6ebe5802e69b699cfbbb05452e0c21b78b224336e24314329", jdbc_driver_class=>"com.microsoft.sqlserver.jdbc.SQLServerDriver", lowercase_column_names=>false, enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_a62e2a3c-81f2-438c-bd8e-f39ad6ec7c6e", enable_metric=>true, charset=>"UTF-8">, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"C:\\Users\\Administrator/.logstash_jdbc_last_run", use_prepared_statements=>false>
  Error: invalid byte sequence in UTF-8
  Exception: ArgumentError
  Stack: org/jruby/RubyString.java:4957:in `partition'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:613:in `placeholder_literal_string_sql_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/sql.rb:112:in `to_s_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1249:in `literal_expression_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:89:in `literal_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1588:in `static_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:236:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/utils/emulate_offset_with_row_number.rb:45:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/shared/mssql.rb:666:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:147:in `sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1602:in `subselect_sql_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1224:in `literal_dataset_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:109:in `literal_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:271:in `aliased_expression_sql_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/sql.rb:112:in `to_s_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1249:in `literal_expression_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:89:in `literal_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1117:in `identifier_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1127:in `block in identifier_list_append'
org/jruby/RubyArray.java:1809:in `each'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1125:in `identifier_list_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1559:in `source_list_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1408:in `select_from_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:245:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/utils/emulate_offset_with_row_number.rb:45:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/shared/mssql.rb:666:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/actions.rb:734:in `single_value!'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/actions.rb:110:in `count'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/extensions/pagination.rb:58:in `each_page'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/plugin_mixins/jdbc/statement_handler.rb:35:in `perform_query'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/plugin_mixins/jdbc/jdbc.rb:201:in `execute_statement'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/inputs/jdbc.rb:316:in `execute_query'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/inputs/jdbc.rb:284:in `run'
C:/java/ELK/logstash-7.9.3/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'
C:/java/ELK/logstash-7.9.3/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'
[2021-02-20T16:03:16,105][INFO ][logstash.inputs.jdbc     ][main][02673f0387aae5f6ebe5802e69b699cfbbb05452e0c21b78b224336e24314329] (0.000600s) SELECT CAST(SERVERPROPERTY('ProductVersion') AS varchar)
[2021-02-20T16:03:16,119][ERROR][logstash.javapipeline    ][main][02673f0387aae5f6ebe5802e69b699cfbbb05452e0c21b78b224336e24314329] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"sa", charset=>"UTF-8", jdbc_paging_enabled=>true, jdbc_page_size=>50000, statement_filepath=>"C:\\java\\ELK\\logstash-7.9.3\\sqlserver\\jdbc.sql", tracking_column_type=>"numeric", type=>"advancelist", record_last_run=>true, use_column_value=>true, tracking_column=>"ID", jdbc_password=><password>, clean_run=>false, jdbc_default_timezone=>"Asia/Shanghai", jdbc_driver_library=>"C:\\java\\ELK\\logstash-7.9.3\\sqlserver\\sqljdbc4.jar", jdbc_connection_string=>"jdbc:sqlserver://127.0.0.1:1433;DatabaseName=PKI", id=>"02673f0387aae5f6ebe5802e69b699cfbbb05452e0c21b78b224336e24314329", jdbc_driver_class=>"com.microsoft.sqlserver.jdbc.SQLServerDriver", lowercase_column_names=>false, enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_a62e2a3c-81f2-438c-bd8e-f39ad6ec7c6e", enable_metric=>true, charset=>"UTF-8">, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"C:\\Users\\Administrator/.logstash_jdbc_last_run", use_prepared_statements=>false>
  Error: invalid byte sequence in UTF-8
  Exception: ArgumentError
  Stack: org/jruby/RubyString.java:4957:in `partition'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:613:in `placeholder_literal_string_sql_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/sql.rb:112:in `to_s_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1249:in `literal_expression_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:89:in `literal_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1588:in `static_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:236:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/utils/emulate_offset_with_row_number.rb:45:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/shared/mssql.rb:666:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:147:in `sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1602:in `subselect_sql_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1224:in `literal_dataset_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:109:in `literal_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:271:in `aliased_expression_sql_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/sql.rb:112:in `to_s_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1249:in `literal_expression_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:89:in `literal_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1117:in `identifier_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1127:in `block in identifier_list_append'
org/jruby/RubyArray.java:1809:in `each'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1125:in `identifier_list_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1559:in `source_list_append'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:1408:in `select_from_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/sql.rb:245:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/utils/emulate_offset_with_row_number.rb:45:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/adapters/shared/mssql.rb:666:in `select_sql'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/actions.rb:734:in `single_value!'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/dataset/actions.rb:110:in `count'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/sequel-5.37.0/lib/sequel/extensions/pagination.rb:58:in `each_page'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/plugin_mixins/jdbc/statement_handler.rb:35:in `perform_query'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/plugin_mixins/jdbc/jdbc.rb:201:in `execute_statement'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/inputs/jdbc.rb:316:in `execute_query'
C:/java/ELK/logstash-7.9.3/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.6/lib/logstash/inputs/jdbc.rb:284:in `run'
C:/java/ELK/logstash-7.9.3/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'
C:/java/ELK/logstash-7.9.3/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'
已邀请:

jiping - 90年后IT男

赞同来自:

问题解决了,
原来是查询脚本中有中文,这也不行???

要回复问题请先登录注册