http client throws error with no content-type header in http response

Bug #996593 reported by Dennis Knochenwefel on 2012-05-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Zorba
Medium
Paul J. Lucas

Bug Description

in modules/com/zorba-xquery/www/modules/http-client.xq.src/http_response_parser.cpp the function HttpResponseParser::parse() throws an error if the response doesn't contain a content-type. In this case theCurrentCharset is "".

the surprising part is that in the following code transcode::is_necessary("") returns true and an icu_streambuf is created internally with an empty charset:

if ( transcode::is_necessary( theCurrentCharset.c_str() ) ) {
  lStream.reset(
    new transcode::stream<std::istream>(
      theCurrentCharset.c_str(), theStreamBuffer
    )
  );
}

in the constructor of the icu_streambuf (src/util/icu_streambuf.cpp) the following code throws the error:

external_conv_( no_conv_ ? nullptr : create_conv( charset ) )

the correct behavior would be if the http-client would not throw an error if the content-type header is missing.

Related branches

Paul J. Lucas (paul-lucas) wrote :

Are you also saying that the correct behavior would also be to not do transcoding and just hope that the unspecified character set is UTF-8?

a comment in the http_response_parser.cpp says that the HTTP/1.1 spec says that the default charset is ISO-8859-1. That is the charset used in the response parser if a content-type without charset is returned.

Paul J. Lucas (paul-lucas) wrote :

When I run the sample query, I get:

</Users/pjl/src/flwor/zorba/repo/bug-996593/zorba/build/URI_PATH/org/expath/ns/http-client.xq>:108,1: static error [err:XQST0059]: "http://www.zorba-xquery.com/modules/converters/html-options": target namespace not found for schema/module "http://www.zorba-xquery.com/modules/converters/html-options.xsd": Could not create stream resource; raised at /Users/pjl/src/flwor/zorba/repo/bug-996593/zorba/src/compiler/translator/translator.cpp:2069

Paul J. Lucas (paul-lucas) wrote :

Also, what should happen if a Content-Type and charset are provided, but the charset is not supported by ICU?

Changed in zorba:
status: New → In Progress
Changed in zorba:
status: In Progress → Opinion

      throw XQUERY_EXCEPTION(
          zerr::ZXQP0006_UNKNOWN_ENCODING,
          ERROR_PARAMS( lEncodingStr.c_str() ),
          ERROR_LOC( loc )
        );

Paul J. Lucas (paul-lucas) wrote :

@Matthias: that code works only for the core and core modules, not modules like the HTTP module.

I know but we can still raise zerr:ZXQP0006 from the http-client module.

Paul J. Lucas (paul-lucas) wrote :

Still waiting for Dennis to give me a working query to test....

> When I run the sample query, I get:
>
> </Users/pjl/src/flwor/zorba/repo/bug-996593/zorba/build/URI_PATH/org/expath/ns
> /http-client.xq>:108,1: static error [err:XQST0059]: "http://www.zorba-
> xquery.com/modules/converters/html-options": target namespace not found
> for schema/module "http://www.zorba-xquery.com/modules/converters/html-
> options.xsd": Could not create stream resource; raised at
> /Users/pjl/src/flwor/zorba/repo/bug-996593/zorba/src/compiler/translator/translator.cpp:2069
>

There is a dependency of the external http-client module to the
data-converters module:

FIND_PACKAGE(zorba_data-converters_module REQUIRED HINTS
${DATA_CONVERTERS_BUILD_DIR})

In addition to the http-client I think you need to check it out to

/Users/pjl/src/flwor/zorba/repo/bug-996593/zorba_modules

I just double checked and the query works on ubuntu 12.04. with the data-converters module installed.

Changed in zorba:
status: Opinion → Fix Committed
milestone: none → 2.5
Changed in zorba:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers