-
Notifications
You must be signed in to change notification settings - Fork 426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PreparedStatement Bulkcopy INSERT issue #2564
Comments
This does not look to be a driver issue. The BatchUpdateException "The system received an invalid column length from the BCP client for the following column ID: 118" There are some discussions about this kind of issues. https://learn.microsoft.com/en-us/answers/questions/1022553/sql-bulk-copy-failed |
Let me note, that the issue does not arise with older jdbc drivers, or if the bulkcopy is switched off. It must have something to do with the driver. |
Hi @dorzsboss, Can you share a full sample record or two to insert into this table g3_t1 that is causing the problem? |
I don't add insert as it is a PreparedStatement and it has to be tested, not a normal insert. But I add a csv file with three row of databulkcopyproblem.csv |
Hi @dorzsboss , This column is defined as Value_date FLOAT(25) And the value in the given csv file is '2024-11-30' Can you please provide a create table, insert table and data file that we can use for reproducing the problem? |
No, you are wrong. The error message claims the problem arises at column 118 as I copyed to the description. And the 118th column is never the value_date which is the second(!) but the txt1. In the description I gave the create statement, please check it again! insert into g3_t1 Are you sure you read the .csv file correctly? I guess the reason you cannot insert right at the second column is because you read the date wrong at column two. But it has nothing to do with the original problem. |
Hi @dorzsboss We understand the issue you have mentioned and need you help in reproducing it and understanding it further. Can you please confirm the following for us, to be able to reproduce it ? If you see any discrepancies in the repro streps below, could you please make corrections and let us know? Create table statement
insert statement
CSV file content
|
Unfortunately my interface program translates the value_date to float. CreateStatement: create table g3_t1 (CUSTOMER_ID VARCHAR(32) ,default FLOAT(25) ,J_termek1 VARCHAR(11) ,J_termek2 VARCHAR(11) |
Driver version
9.2.1 and later
SQL Server version
Microsoft SQL Server 2016 (SP3-CU1-GDR) (KB5046856) - 13.0.7050.2 (X64) Oct 14 2024 22:01:52 Copyright (c) Microsoft Corporation
Enterprise Edition: Core-based Licensing (64-bit) on Windows Server 2016 Standard 10.0 (Build 14393: ) (Hypervisor)
Client Operating System
Linux RedHat
JAVA/JVM version
1.8.0
Table schema
Provide the table schema to repro the issue.
create table g3_t1 (CUSTOMER_ID VARCHAR(32) ,Value_date FLOAT(25) ,_default FLOAT(25) ,J_termek1 VARCHAR(11)
,J_termek2 VARCHAR(11) ,J_termek1_appl_pont FLOAT(25) ,J_termek1_beh_pont FLOAT(25) ,J_termek2_appl_pont FLOAT(25) ,J_termek
2_beh_pont FLOAT(25) ,J_termek1_appl_intercept FLOAT(25) ,J_termek1_appl_beta FLOAT(25) ,J_termek1_beh_intercept FLOAT(25)
,J_termek1_beh_beta FLOAT(25) ,J_termek1_appl_weight FLOAT(25) ,J_termek1_beh_weight FLOAT(25) ,J_REL_def_STD FLOAT(25) ,J_macr
o_correction FLOAT(25) ,J_termek2_appl_intercept FLOAT(25) ,J_termek2_appl_beta FLOAT(25) ,J_termek2_beh_intercept FLOAT(25)
,J_termek2_beh_beta FLOAT(25) ,J_termek2_appl_weight FLOAT(25) ,J_termek2_beh_weight FLOAT(25) ,J_termek1_weight FLOAT(25) ,J
termek2_weight FLOAT(25) ,L_termek1 VARCHAR(11) ,L_termek2 VARCHAR(11) ,L_termek1_appl_pont FLOAT(25) ,L_termek1_beh_pont FLOAT(25)
,L_termek2_appl_pont FLOAT(25) ,L_termek2_beh_pont FLOAT(25) ,L_termek1_appl_intercept FLOAT(25) ,L_termek1_appl_beta
FLOAT(25) ,L_termek1_beh_intercept FLOAT(25) ,L_termek1_beh_beta FLOAT(25) ,L_termek1_appl_weight FLOAT(25) ,L_termek1_beh_weight
FLOAT(25) ,L_REL_def_STD FLOAT(25) ,L_macro_correction FLOAT(25) ,L_termek2_appl_intercept FLOAT(25) ,L_termek2_appl_be
ta FLOAT(25) ,L_termek2_beh_intercept FLOAT(25) ,L_termek2_beh_beta FLOAT(25) ,L_termek2_appl_weight FLOAT(25) ,L_termek2_beh_weight
FLOAT(25) ,L_termek1_weight FLOAT(25) ,L_termek2_weight FLOAT(25) ,S_termek1 VARCHAR(11) ,S_termek2 VARCHAR(11) ,S_te
rmek1_appl_pont FLOAT(25) ,S_termek1_beh_pont FLOAT(25) ,S_termek2_appl_pont FLOAT(25) ,S_termek2_beh_pont FLOAT(25)
,S_termek1_appl_intercept FLOAT(25) ,S_termek1_appl_beta FLOAT(25) ,S_termek1_beh_intercept FLOAT(25) ,S_termek1_beh_beta FLOAT(25) ,
S_termek1_appl_weight FLOAT(25) ,S_termek1_beh_weight FLOAT(25) ,S_REL_def_STD FLOAT(25) ,S_macro_correction FLOAT(25)
,S_termek2_appl_intercept FLOAT(25) ,S_termek2_appl_beta FLOAT(25) ,S_termek2_beh_intercept FLOAT(25) ,S_termek2_beh_beta FLOAT(25)
,S_termek2_appl_weight FLOAT(25) ,S_termek2_beh_weight FLOAT(25) ,S_termek1_weight FLOAT(25) ,S_termek2_weight FLOAT(25) ,O_termek1
VARCHAR(11) ,O_termek2 VARCHAR(11) ,O_termek1_appl_pont FLOAT(25) ,O_termek1_beh_pont FLOAT(25) ,O_termek2_appl_pont
FLOAT(25) ,O_termek2_beh_pont FLOAT(25) ,O_termek1_appl_intercept FLOAT(25) ,O_termek1_appl_beta FLOAT(25) ,O_termek1_beh_intercept
FLOAT(25) ,O_termek1_beh_beta FLOAT(25) ,O_termek1_appl_weight FLOAT(25) ,O_termek1_beh_weight FLOAT(25) ,O_REL_def_ST
D FLOAT(25) ,O_macro_correction FLOAT(25) ,O_termek2_appl_intercept FLOAT(25) ,O_termek2_appl_beta FLOAT(25) ,O_termek2_beh_intercept
FLOAT(25) ,O_termek2_beh_beta FLOAT(25) ,O_termek2_appl_weight FLOAT(25) ,O_termek2_beh_weight FLOAT(25) ,O_termek1
weight FLOAT(25) ,O_termek2_weight FLOAT(25) ,H_termek1 VARCHAR(11) ,H_termek2 VARCHAR(11) ,H_termek1_appl_pont FLOAT(25)
,H_termek1_beh_pont FLOAT(25) ,H_termek2_appl_pont FLOAT(25) ,H_termek2_beh_pont FLOAT(25) ,H_termek1_appl_intercept FLOAT(25) ,
H_termek1_appl_beta FLOAT(25) ,H_termek1_beh_intercept FLOAT(25) ,H_termek1_beh_beta FLOAT(25) ,H_termek1_appl_weight FLOAT(25)
,H_termek1_beh_weight FLOAT(25) ,H_REL_def_STD FLOAT(25) ,H_macro_correction FLOAT(25) ,H_termek2_appl_intercept FLOAT(25)
,H_termek2_appl_beta FLOAT(25) ,H_termek2_beh_intercept FLOAT(25) ,H_termek2_beh_beta FLOAT(25) ,H_termek2_appl_weight FLOAT(25)
,H_termek2_beh_weight FLOAT(25) ,H_termek1_weight FLOAT(25) ,H_termek2_weight FLOAT(25) ,H_atlag_pd FLOAT(25) ,spare3 VA
RCHAR(26) ,BEH_PD FLOAT(25) ,behpd FLOAT(25) ,txt1 VARCHAR(10) ,szam FLOAT(25) )
Problem description
I try to insert rows into this table ( setUseBulkCopyForBatchInsert=true ) and I got this error:
java.sql.BatchUpdateException: A rendszer érvénytelen oszlophosszt kapott a BCP-ügyféltől a következő oszlopazonosítóhoz: 118. at
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:2042) at
SAS2SQL.writeToOutput(SAS2SQL.java:1098) at
SAS2SQL.main(SAS2SQL.java:121)
Its in hungarian, in english it means that the system got an invalid column length at 118 which is the txt1 column.
If I replace this column with any kind of numeric field, the problem disappears but appears with text columns. It doesn't depend on the column is VARCHAR or NVARCHAR and has nothing to do with encoding. Neither it depends on the value or the length of the value I try to load into this column, "abc", "", "123423543" all produces the error, right at the first line. The error disappears if I set setUseBulkCopyForBatchInsert=false so it is related to the bulkcopy engine.
Expected behavior
I expect the the insert happening without issues with setUseBulkCopyForBatchInsert=true setting regardless what kind of column is the last of this table .
#Output of what you actually see.
I am on the client side, an can see only the java log.
Any other details that can be helpful
A lot.
My PreparedStatement:
insert into g3_t1
(CUSTOMER_ID,Value_date,default,J_termek1,J_termek2,J_termek1_appl_pont,J_termek1_beh_pont,J_termek2_appl_pont,J_termek2_beh_pont,J_t
ermek1_appl_intercept,J_termek1_appl_beta,J_termek1_beh_intercept,J_termek1_beh_beta,J_t
ermek1_appl_weight,J_termek1_beh_weight,J_REL_def_STD,J_macro_correction,J_termek2_appl_intercept,J_termek2_appl_beta,J_termek2_beh_in
tercept,J_termek2_beh_beta,J_termek2_appl_weight,J_termek2_beh_weight,J_termek1_weight,J_termek2_weight,L_termek1,L
termek2,L_termek1_appl_pont,L_termek1_beh_pont,L_termek2_appl_pont,L_termek2_beh_pont,L_termek1_appl_intercept,L_termek1_appl_beta,L_t
ermek1_beh_intercept,L_termek1_beh_beta,L_termek1_appl_weight,L_termek1_beh_weight,L_REL_def_STD,L_macro_correction,
L_termek2_appl_intercept,L_termek2_appl_beta,L_termek2_beh_intercept,L_termek2_beh_beta,L_termek2_appl_weight,L_termek2_beh_weight,L_t
ermek1_weight,L_termek2_weight,S_termek1,S_termek2,S_termek1_appl_pont,S_termek1_beh_pont,S_termek2_appl_pont,S_term
ek2_beh_pont,S_termek1_appl_intercept,S_termek1_appl_beta,S_termek1_beh_intercept,S_termek1_beh_beta,S_termek1_appl_weight,S_termek1_b
eh_weight,S_REL_def_STD,S_macro_correction,S_termek2_appl_intercept,S_termek2_appl_beta,S_termek2_beh_intercept,S_te
rmek2_beh_beta,S_termek2_appl_weight,S_termek2_beh_weight,S_termek1_weight,S_termek2_weight,O_termek1,O_termek2,O_termek1_appl_pont,O_
termek1_beh_pont,O_termek2_appl_pont,O_termek2_beh_pont,O_termek1_appl_intercept,O_termek1_appl_beta,O_termek1_beh_i
ntercept,O_termek1_beh_beta,O_termek1_appl_weight,O_termek1_beh_weight,O_REL_def_STD,O_macro_correction,O_termek2_appl_intercept,O_ter
mek2_appl_beta,O_termek2_beh_intercept,O_termek2_beh_beta,O_termek2_appl_weight,O_termek2_beh_weight,O_termek1_weigh
t,O_termek2_weight,H_termek1,H_termek2,H_termek1_appl_pont,H_termek1_beh_pont,H_termek2_appl_pont,H_termek2_beh_pont,H_termek1_appl_in
tercept,H_termek1_appl_beta,H_termek1_beh_intercept,H_termek1_beh_beta,H_termek1_appl_weight,H_termek1_beh_weight,H_
REL_def_STD,H_macro_correction,H_termek2_appl_intercept,H_termek2_appl_beta,H_termek2_beh_intercept,H_termek2_beh_beta,H_termek2_appl_
weight,H_termek2_beh_weight,H_termek1_weight,H_termek2_weight,H_atlag_pd,spare3,BEH_PD,behpd,txt1) VALUES
(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?
,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
The crucial part of my java code (textelem is a string value I want to load into the columns) :
if (textelem==null)pstmt.setString(k,null);
else {
if (tip==Types.VARCHAR || tip==Types.NVARCHAR ) {
pstmt.setString(k, textelem);
insertedRows++;
pstmt.addBatch();
bulkcounter++;
if (bulkcounter==batchSize){
pstmt.executeBatch();
pstmt.clearBatch();
bulkcounter=0;
if (monitor)log(getTime()+" / "+insertedRows);
}
JDBC trace logs
java.sql.BatchUpdateException: A rendszer érvénytelen oszlophosszt kapott a BCP-ügyféltől a következő oszlopazonosítóhoz: 118. at
com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:2042) at
SAS2SQL.writeToOutput(SAS2SQL.java:1098) at
SAS2SQL.main(SAS2SQL.java:121)
The text was updated successfully, but these errors were encountered: