Bug #2043437 “Three test failures on rel_3_12 and main” : Bugs : Evergreen

Revision history for this message

Jason Stephenson (jstephenson) wrote on 2023-11-14:

#1

I'll take a look at the Perl search test.

Revision history for this message

Jason Stephenson (jstephenson) wrote on 2023-11-14:

#2

Download full text (7.3 KiB)

On the Perl test failure, The storage query below, triggers the following error:

ERROR: syntax error
CONTEXT: PL/pgSQL function query_int_wrapper(integer[],text) line 3 at RETURN

WITH w AS (

WITH x55a483de3858_keyword_xq AS (SELECT
      ((to_tsquery('simple', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|') || ')', '()'), '')) || to_tsquery('english_nostop', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|') || ')', '()'), '')))) AS tsq,
      (to_tsquery('simple', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|') || ')', '()'), '')) || to_tsquery('english_nostop', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|') || ')', '()'), ''))) AS tsq_rank ),
x55a483de3858_keyword AS (
          SELECT fe.*, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* search */
            FROM metabib.keyword_field_entry AS fe
              JOIN config.metabib_field AS fe_weight ON (fe_weight.id = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM metabib.keyword_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (39,41,42,46,47,48,50) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM metabib.subject_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (16) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM metabib.title_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (6,53) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM metabib.author_field_entry AS fe
           ...

On the Perl test failure, The storage query below, triggers the following error:

ERROR:  syntax error
CONTEXT:  PL/pgSQL function query_int_wrapper(integer[],text) line 3 at RETURN

WITH w AS (

WITH x55a483de3858_keyword_xq AS (SELECT 
      ((to_tsquery('simple', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|')  || ')',  '()'), '')) || to_tsquery('english_nostop', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|')  || ')',  '()'), '')))) AS tsq,
      (to_tsquery('simple', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|')  || ')',  '()'), '')) || to_tsquery('english_nostop', COALESCE(NULLIF( '(' || btrim(regexp_replace(search_normalize(split_date_range($_39864$dragons$_39864$)),E'(?:\\s+|:)','&','g'),'&|')  || ')',  '()'), ''))) AS tsq_rank ),
 x55a483de3858_keyword AS (
          SELECT fe.*, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* search */
            FROM  metabib.keyword_field_entry AS fe
              JOIN config.metabib_field AS fe_weight ON (fe_weight.id = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM  metabib.keyword_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (39,41,42,46,47,48,50) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM  metabib.subject_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (16) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM  metabib.title_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (6,53) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)
                UNION ALL
          SELECT fe.id, fe.source, fe.field, fe.value, fe.index_vector, fe_weight.weight, x55a483de3858_keyword_xq.tsq, x55a483de3858_keyword_xq.tsq_rank /* virtual field addition */
            FROM  metabib.author_field_entry AS fe
              JOIN config.metabib_field_virtual_map AS fe_weight ON (fe_weight.virtual = 45 AND fe_weight.real IN (8) AND fe_weight.real = fe.field)
              JOIN x55a483de3858_keyword_xq ON (fe.index_vector @@ x55a483de3858_keyword_xq.tsq)),lang_with AS (SELECT id FROM config.coded_value_map WHERE ctype = 'item_lang' AND code = $_39864$eng$_39864$),        pop_with AS (
            SELECT  record,
                    ARRAY_AGG(badge) AS badges,
                    SUM(s.score::NUMERIC*b.weight::NUMERIC)/SUM(b.weight::NUMERIC) AS total_score
              FROM  rating.record_badge_score s
                    JOIN rating.badge b ON (
                        b.id = s.badge
 AND b.scope = ANY ('{1}')) GROUP BY 1)
,c_attr AS (SELECT (ARRAY_TO_STRING(ARRAY[c_attrs,search.calculate_visibility_attribute_test('circ_lib','{1,2,3,4,5,6,7,8,9}',FALSE)],'&'))::query_int AS vis_test FROM asset.patron_default_visibility_mask() x OFFSET 0)
,b_attr AS (SELECT (b_attrs||search.calculate_visibility_attribute_test('luri_org','{1}',FALSE))::query_int AS vis_test FROM asset.patron_default_visibility_mask() x OFFSET 0)
SELECT  id,
        rel,
        CASE WHEN cardinality(records) = 1 THEN records[1] ELSE NULL END AS record,
        NULL::INT AS total,
        NULL::INT AS checked,
        NULL::INT AS visible,
        NULL::INT AS deleted,
        NULL::INT AS excluded,
        badges,
        popularity
  FROM  (SELECT m.source AS id,
                ARRAY[m.source] AS records,
                (AVG(
          (COALESCE(ts_rank_cd('{0.1, 0.2, 0.4, 1.0}', x55a483de3858_keyword.index_vector, x55a483de3858_keyword.tsq_rank, 14) * x55a483de3858_keyword.weight * 1000, 0.0)
          * COALESCE(evergreen.rel_bump(('{' || quote_literal(search_normalize($_39864$dragons$_39864$)) || '}')::TEXT[], x55a483de3858_keyword.value, '{word_order}'::TEXT[], '{10}'::NUMERIC[]),1.0))
        )+1 * COALESCE( NULLIF( FIRST(mrv.vlist @> ARRAY[lang_with.id]), FALSE )::INT * 5, 1))::NUMERIC AS rel,
                1.0/((AVG(
          (COALESCE(ts_rank_cd('{0.1, 0.2, 0.4, 1.0}', x55a483de3858_keyword.index_vector, x55a483de3858_keyword.tsq_rank, 14) * x55a483de3858_keyword.weight * 1000, 0.0)
          * COALESCE(evergreen.rel_bump(('{' || quote_literal(search_normalize($_39864$dragons$_39864$)) || '}')::TEXT[], x55a483de3858_keyword.value, '{word_order}'::TEXT[], '{10}'::NUMERIC[]),1.0))
        )+1 * COALESCE( NULLIF( FIRST(mrv.vlist @> ARRAY[lang_with.id]), FALSE )::INT * 5, 1)))::NUMERIC AS rank, 
                FIRST(pubdate_t.value) AS tie_break,
                STRING_AGG(ARRAY_TO_STRING(pop_with.badges,','),',') AS badges,
                AVG(COALESCE(pop_with.total_score::NUMERIC,0.0::NUMERIC))::NUMERIC(2,1) AS popularity
          FROM  metabib.metarecord_source_map m
                
        LEFT JOIN x55a483de3858_keyword ON (m.source = x55a483de3858_keyword.source)
                
                INNER JOIN metabib.record_attr_vector_list mrv ON m.source = mrv.source
                INNER JOIN biblio.record_entry bre ON m.source = bre.id AND NOT bre.deleted
                 INNER JOIN asset.course_module_course_materials acmcm ON m.source = acmcm.record INNER JOIN asset.course_module_course acmc ON acmcm.course=acmc.id 
                LEFT JOIN pop_with ON ( m.source = pop_with.record )
                LEFT JOIN metabib.record_sorter pubdate_t ON m.source = pubdate_t.source AND attr = 'pubdate'
                ,lang_with
                ,c_attr
                ,b_attr 
          WHERE 1=1
                AND (
          (x55a483de3858_keyword.id IS NOT NULL)
          AND evergreen.query_int_wrapper(mrv.vlist, '()')
        )
        AND (
          (EXISTS (SELECT 1 FROM asset.copy_vis_attr_cache WHERE record = m.source AND vis_attr_vector @@ c_attr.vis_test)) OR ((b_attr.vis_test IS NULL OR bre.vis_attr_vector @@ b_attr.vis_test))
        )
                
          GROUP BY 1
          ORDER BY 4 ASC NULLS LAST,  5 DESC NULLS LAST, 3 DESC
          LIMIT 100000
        ) AS core_query
) (SELECT * FROM w LIMIT 1000 OFFSET 0)
        UNION ALL
  SELECT NULL,NULL,NULL,COUNT(*),COUNT(*),COUNT(*),0,0,NULL,NULL FROM w;

I don't know how to fix this. It looks like a bug in the query_int_wrapper function.

Revision history for this message

Jane Sandberg (sandbergja) wrote on 2023-11-15:

#3

Per Jason's recommendation, I'm starting a collab branch. This includes the fix to #1.

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/sandbergja/lp2043437-3.12-test-failures

Fixes to #2 and #3 very welcome!

Revision history for this message

Jane Sandberg (sandbergja) wrote on 2023-11-16 (last edit on 2023-11-16):

#4

I pushed patches for #2 and #3 to https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/sandbergja/lp2043437-3.12-test-failures.

The fix for #3 is just reverting commit 828a6f.

To test, install Evergreen with this patch, and:
1. Run the e2e tests, and confirm that they pass
2. Run the perl live tests, and confirm that they pass
3. Run a bunch of searches with or without the course materials filter. Make sure that they are all performant and return the expected results.
4. Open a bib record, and confirm that you can open the Holdings View page.

tags:

added: pullrequest

Galen Charlton (gmc) on 2023-11-16

Changed in evergreen:
assignee:	nobody → Galen Charlton (gmc)
milestone:	none → 3.12-beta
status:	New → Confirmed

Revision history for this message

Galen Charlton (gmc) wrote on 2023-11-17:

#5

Tested and pushed to main and rel_3_12. Thanks, Jane!

Changed in evergreen:
status:	Confirmed → Fix Committed
assignee:	Galen Charlton (gmc) → nobody

Evergreen Bug Maintenance (bugmaster) on 2023-11-20

Changed in evergreen:
status:	Fix Committed → Fix Released

Evergreen

Three test failures on rel_3_12 and main

Bug Description

Other bug subscribers

Remote bug watches