Jack Luo
05/21/2024, 10:59 PMsrc/main/java/org/apache/pinot/query/parser/ParserUtils.java . Obviously, we can hard-code a larger max length which is fine with us. For OSS, is there any recommendation on the mechiansm to pass in this max length as a config property?Jack Luo
05/22/2024, 7:24 PMJack Luo
05/23/2024, 8:40 PMJack Luo
05/24/2024, 6:47 PMParserUtils.java is actually dead code and the code that actually gets used is in pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java. Specifically,
static SqlParserImpl newSqlParser(StringReader inStream) {
SqlParserImpl sqlParser = new SqlParserImpl(inStream);
sqlParser.switchTo(SqlAbstractParserImpl.LexicalState.DQID);
// TODO: convert to MySQL conformance once we retired most of the un-tested BABEL tokens
sqlParser.setConformance(SqlConformanceEnum.BABEL);
sqlParser.setTabSize(1);
sqlParser.setQuotedCasing(Casing.UNCHANGED);
sqlParser.setUnquotedCasing(Casing.UNCHANGED);
sqlParser.setIdentifierMaxLength(<some configurable value>); <---- can add config here.
return sqlParser;
}
If we want to make it configurable, adding a static variable, and setting it in some configuration parsing place could work. So any recommendation on a good place to specify the max identifier length should be a sufficient guidance.