Import schema segfaults

[William Candillon]

Starting program: /Users/wcandillon/28msec/zorba/build/dist/bin/zorba -q CheckTaxonomy.xq -f
Reading symbols for shared libraries ++++++++............................................................... done
Reading symbols for shared libraries . done
Reading symbols for shared libraries ........ done
Reading symbols for shared libraries ... done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: 13 at address: 0x0000000000000000
0x00007fff87b4b062 in std::istream::sentry::sentry ()
(gdb) where
#0 0x00007fff87b4b062 in std::istream::sentry::sentry ()
#1 0x00007fff87b4b553 in std::istream::read ()
#2 0x000000010162168d in zorba::IstreamBinInputStream::readBytes ()
#3 0x0000000103d921e3 in xercesc_2_8::XMLReader::refreshRawBuffer ()
#4 0x0000000103d93b47 in xercesc_2_8::XMLReader::XMLReader ()
#5 0x0000000103d93c39 in xercesc_2_8::XMLReader::XMLReader ()
#6 0x0000000103d17eda in xercesc_2_8::ReaderMgr::createReader ()
#7 0x0000000103cef908 in xercesc_2_8::IGXMLScanner::scanReset ()
#8 0x0000000103cedc18 in xercesc_2_8::IGXMLScanner::scanDocument ()
#9 0x0000000103c66998 in xercesc_2_8::AbstractDOMParser::parse ()
#10 0x0000000103d58c77 in xercesc_2_8::TraverseSchema::preprocessImport ()
#11 0x0000000103d57e72 in xercesc_2_8::TraverseSchema::preprocessChildren ()
#12 0x0000000103d58690 in xercesc_2_8::TraverseSchema::preprocessSchema ()
#13 0x0000000103d6d899 in xercesc_2_8::TraverseSchema::TraverseSchema ()
#14 0x0000000103d6d972 in xercesc_2_8::TraverseSchema::TraverseSchema ()
#15 0x0000000103d2eb5d in xercesc_2_8::SGXMLScanner::loadXMLSchemaGrammar ()
#16 0x0000000103d2ee01 in xercesc_2_8::SGXMLScanner::loadGrammar ()
#17 0x0000000103d9a0e5 in xercesc_2_8::XMLScanner::loadGrammar ()
#18 0x0000000103d9a1a7 in xercesc_2_8::XMLScanner::loadGrammar ()
#19 0x0000000103d2879a in xercesc_2_8::SAX2XMLReaderImpl::loadGrammar ()
#20 0x0000000101614c4d in zorba::Schema::registerXSD ()
#21 0x0000000100347e7a in zorba::TranslatorImpl::import_schema ()
#22 0x0000000100290924 in zorba::TranslatorImpl::begin_visit ()
#23 0x000000010019f6e0 in zorba::SchemaImport::accept ()
#24 0x00000001001ab8ac in zorba::SIND_DeclList::accept ()
#25 0x000000010019f52e in zorba::Prolog::accept ()
#26 0x000000010019f21d in zorba::MainModule::accept ()
#27 0x0000000100214d7d in zorba::translate_aux ()
#28 0x00000001002148eb in zorba::translate ()
#29 0x00000001005d87c6 in zorba::XQueryCompiler::normalize ()
#30 0x00000001005d89c3 in zorba::XQueryCompiler::compile ()
#31 0x00000001005da700 in zorba::XQueryCompiler::compile ()
#32 0x000000010004ba8a in zorba::XQueryImpl::doCompile ()
#33 0x000000010004c796 in zorba::XQueryImpl::compile ()
#34 0x00000001000053e8 in compileAndExecute ()
#35 0x0000000100009a87 in main ()

[William Candillon (wcandillon)]

Here's the query: http://dl.dropbox.com/u/1487285/DimensionsValidationXQuery.zip

[Cezar Andrei (cezar-andrei)]

If I run the original XBRL scripts without any modifications, I get the following kind of error. It's not always the same error but it seems that the error comes from a file downloaded from the net (by import schema ... at "http://..." ) and the file has utf8 errors inside. So there are 2 possible bugs: - 1 on the server side the served file for our User-Agent (looks fine on firefox) is broken, - or 2 on our side the download code doesn't save the right bytes.

Running Zorba ...

</media/sdd/dev/xqSample/xbrl-orig/CheckTaxonomy.xq>:38,1: static error [err:XQST0059]: "Fatal error in schema with system id http://www.xbrl.org/2003/xl-2003-12-31.xsd, line 1, column 1.
invalid multi-byte sequence": target namespace not found for schema/module; raised at /home/cezar/dev/zorba/src/types/schema/LoadSchemaErrorHandler.cpp:88
Done

If a run the modified scripts so that it doesn't go on the net to download files, I get another error, see below. There seems to be a semantic error a file is expected at a location but it's not available.

cezar@i7:~/dev/xqSample/xbrl-modified$ ./run.sh 2>&1

Running Zorba ...

<?xml version="1.0" encoding="UTF-8"?>
 -- startingDocument: [0]: xs:anyURI(file:///media/sdd/dev/xqSample/xbrl-modified/100-xbrldte/101-HypercubeElementIsNotAbstractError/hypercubeValid.xsd)
File: [0]: xs:string(100-xbrldte/101-HypercubeElementIsNotAbstractError/hypercubeValid.xsd)
 -- startingDocument: [0]: xs:anyURI(file:///media/sdd/dev/xqSample/xbrl-modified/100-xbrldte/101-HypercubeElementIsNotAbstractError/hypercubeNotAbstract.xsd)
File: [0]: xs:string(100-xbrldte/101-HypercubeElementIsNotAbstractError/hypercubeNotAbstract.xsd)
 -- startingDocument: [0]: xs:anyURI(file:///media/sdd/dev/xqSample/xbrl-modified/100-xbrldte/101-HypercubeElementIsNotAbstractError/hypercubeNotAbstractWithSGComplexities.xsd)
File: [0]: xs:string(100-xbrldte/101-HypercubeElementIsNotAbstractError/hypercubeNotAbstractWithSGComplexities.xsd)
 -- startingDocument: [0]: xs:anyURI(file:///media/sdd/dev/xqSample/xbrl-modified/100-xbrldte/102-HypercubeDimensionSourceError/hypercubeDimensionValid.xsd)
  -- file: [0]: xs:anyURI(file:///media/sdd/dev/xqSample/xbrl-modified/hypercubeDimensionValid-definition.xml)
 -- startingDocument: [0]: xs:anyURI(file:///media/sdd/dev/xqSample/xbrl-modified/hypercubeDimensionValid-definition.xml)
</media/sdd/dev/xqSample/xbrl-modified/xf-2006.xq>:1147,19: dynamic error [err:FODC0002]: "file:///media/sdd/dev/xqSample/xbrl-modified/hypercubeDimensionValid-definition.xml": error retrieving resource; raised at /home/cezar/dev/zorba/src/runtime/sequences/sequences_impl.cpp:1825
=================================================
xfi:DTS-files#2 <http://www.xbrl.org/2005/function/instance>
/media/sdd/dev/xqSample/xbrl-modified/xf-2006.xq at line 1213 column 8
=================================================
xfi:DTS-files#2 <http://www.xbrl.org/2005/function/instance>
/media/sdd/dev/xqSample/xbrl-modified/xf-2006.xq at line 1139 column 2
=================================================
xfi:DTS-files#1 <http://www.xbrl.org/2005/function/instance>
/media/sdd/dev/xqSample/xbrl-modified/XBRLChkDimTx.xq at line 98 column 22
=============...

[Cezar Andrei (cezar-andrei)]

Assigned to Chris to make sure the stream doesn't get corrupted.

[Chris Hillery (ceejatec)]

I'm not able to reproduce any crash, but I can't get the query to run. The query imports the XMLSchema schema directly:

  import schema namespace xs = "http://www.w3.org/2001/XMLSchema" at "http://www.w3.org/2001/XMLSchema.xsd";

and this doesn't seem to work:

</tmp/foo/conformance-dimensions/conf/CheckTaxonomy.xq>:42,1: static error [err:XQST0059]: "Fatal error in schema with system id http://www.w3.org/2001/XMLSchema.xsd, line 1, column 3.
markup declaration expected": target namespace not found for schema/module; raised at /home/ceej/zo/src/src/types/schema/LoadSchemaErrorHandler.cpp:88

I don't know what's going on here, but I did try downloading XMLSchema.xsd and putting it on Zorba's URI path so it would be loaded locally. This led me to also download XMLSchema.dtd and datatypes.dtd which are referenced by XMLSchema.xsd. With all those on the URI path, I now get a bunch of warnings like

Warning at file file:///home/ceej/zo/src/build/URI_PATH/org/w3/www/2001/XMLSchema.xsd, line 346, char 13
  Message: attribute 'id' has already been declared for element 'xs:field'

finally followed by an error:

</tmp/foo/conformance-dimensions/conf/CheckTaxonomy.xq>:42,1: static error [err:XQST0059]: "Error in schema with system id file:///home/ceej/zo/src/build/URI_PATH/org/w3/www/2001/XMLSchema.xsd, line 177, column 47.
type 'http://www.w3.org/2001/XMLSchema:derivationControl' not found": target namespace not found for schema/module; raised at /home/ceej/zo/src/src/types/schema/LoadSchemaErrorHandler.cpp:68

It kind of seems like Zorba has some idea of the contents of XMLSchema.xsd built-in; is that correct? I'm afraid I don't know how to debug this any further, but it doesn't immediately seem to indicate a problem in URI resolution which is why I assume the bug was assigned to me. Does anyone else know how schema-for-schema is "supposed" to work with Zorba?

[Cezar Andrei (cezar-andrei)]

Zorba knows without any import about the built-in schema types but those are only a small subset of schema4schema. Depending on which set is used it is or is not required to import schema4schema.

[Cezar Andrei (cezar-andrei)]

I'll take a look at it.

[Cezar Andrei (cezar-andrei)]

I was able to reproduce it exactly as Chris.
The error is thrown by Xerces, even if the type it complains about 'derivationControl' is defined lower in the same file XMLSchema.xsd at line 2082.

The bug is in Xerces, I filed it in their Jira bug tracker: https://issues.apache.org/jira/browse/XERCESC-1980

[Cezar Andrei (cezar-andrei)]

The import to XMLSchema is not required, since Zorba knows about xs:string and the rest of built-in types used in this query.
If I remove the import of XMLSchema.xsd still fails, but it looks like it's an error in the script logic, it's doing doc($file) for a file that doesn't exist:

 TR [0]: xs:anyURI(file:///home/cezar/dev/xqSample/conformance-dimensions/conf/hypercubeDimensionValid-definition.xml)
</home/cezar/dev/xqSample/conformance-dimensions/conf/xf-2006.xq>:1147,19: dynamic error [err:FODC0002]: "file:///home/cezar/dev/xqSample/conformance-dimensions/conf/hypercubeDimensionValid-definition.xml": error retrieving resource; raised at /media/sdd/dev/repo/bug867156-importsegfaults/src/runtime/sequences/sequences_impl.cpp:1849

[Cezar Andrei (cezar-andrei)]

According to a Xerces developer ( see comment at https://issues.apache.org/jira/browse/XERCESC-1980 ) Xerces should not import the XMLSchema.xsd schema. Fix was added in translator to skip importing it.

Query in conformance-dimentions fails now as in previous comment, it seems that there is a bug in the script logic.