400 bad request from server API

[Justin Marshall]

When trying to POST to the /cdb/set-fields/ endpoint, I am getting a 400 response.

curl -v \
        -X POST \
        --digest -u "user:password" \
        -H "User-Agent: Unknown/Unknown (Unknown; build:Unknown; iOS 12.1.0) Alamofire/4.8.1" \
        -H "Accept-Language: en;q=1.0" \
        -H "Content-Type: application/json" \
        -H "Accept-Encoding: gzip;q=1.0, compress;q=0.5" \
        -d "{"title":"'Salem's LotAPPTEST"}" \
        "http://my.url.here/cdb/set-fields/1/"

Is my JSON body incorrect? I really wish this API were documented...

[Kovid Goyal (kovid)]

Look at the body of the 400 response, that will tell you what the problem
is.

 status invalid

[Kovid Goyal (kovid)]

Also, you dont need to use curl if you are trying to set metadata via
the API, you can use calibredb directly.

[Kovid Goyal (kovid)]

Oh and looking at your curl command-line, the quoting of the -d argument is incorrect.

[Justin Marshall (jdmarshall90)]

The only thing I get in the response body is "Invalid encoded data".

This is just a curl'd version of the request my iOS application is sending.

I tried with and without escape characters, and still get the same response.

[Kovid Goyal (kovid)]

That error indicates your JSON is not valid, and your quoting is
definitely off. You can tjust drop/add the quotes, you have to escape
them properly, the details of which depend on the shell you are using.

[Justin Marshall (jdmarshall90)]

Fair enough. Do you have an example of a valid post body?

My conversion of it to curl from how the iOS networking library (Alamofire) is printing it to the console may be off, so here's the exact version that my app is using:

{\"title\":\"\'Salem\'s LotAPPTEST\"}

This is identical to how Alamofire escapes all of its JSON-encoded requests, in my experience.

[Kovid Goyal (kovid)]

Sorry, I'm not going to struggle with shell escaping, but the JSON that you need, is of the form:

{"changes": {"title": "whatever"}}, "loaded_book_ids": [some_book_id]}

[Justin Marshall (jdmarshall90)]

Ahh, so I was missing some fields. Thank you! I’ll try these inputs.

What does the loaded_book_ids field signify? Isn’t the book id that is being changed already encoded into the url?

[Kovid Goyal (kovid)]

It is the set of books whose metadata the method should return if that
metadata was changed byt he set-field operation. Remember that is you
for instance, change the case of a tag, it will be changed on all books
that have that tag.

[Justin Marshall (jdmarshall90)]

So would it be a best practice to pass in all books with that same tag / author / etc., so that they can be reloaded without a complete refetch of the library?

If that’s the case, I think it would simplify the api if it could just return them all automatically, or clients could just pass an extra flag “returnAllChanges”: true

[Kovid Goyal (kovid)]

It depends on how your app stores the metadata. If it has local copies
of only a subset, then pass in only that subset, if it has all books,
then pass in all book ids. I can certainly add a field to obviate the
necessity of passing in all book ids, but its not that big of a deal to
pass in a list of ids.

[Kovid Goyal (kovid)]

https://github.com/kovidgoyal/calibre/commit/d7d7810a4498e4a62c8fb2a02a433b04404b586c

[Justin Marshall (jdmarshall90)]

Thanks, I updated to use this payload:

{"changes":{"title":"'Salem's LotAPPTESTTAKE2"},"loaded_book_ids":[1]}

And I get a 400 when the server is started via this command:

calibre-server --port=PORT_NUMBER --enable-local-write --num-per-page=500 --userdb /path/to/calibre/users/db/users.sqlite --enable-auth --ban-after=3 --ban-for=60

But when I run the exact same request, pointing to the server started via the GUI, it works fine.

[Kovid Goyal (kovid)]

The server runs the same code when started from the GUI or command-line.

[Justin Marshall (jdmarshall90)]

Regardless, it is responding differently depending on how I start it.

[Justin Marshall (jdmarshall90)]

Long story short, that particular problem was something in my network stack.

I'm now seeing a different issue:

When I pass in the array of "loaded_book_ids", in the response, I get the correct book ids, but the rest of the metadata in those books matches the book that I just changed.

For example, I change book ID 2's "History" tag to "history", and pass in book ids 1, 2, 3, 4. In the response, and I get 4 books back, all matching the book that I was originally changing -- except for the book ids, those are correct.

I set some breakpoints in the Calibre web app's Javascript, and it appears to be behaving the same way.

Am I missing something here?

[Kovid Goyal (kovid)]

Fixed in branch master. The fix will be in the next release. calibre is usually released every alternate Friday.

 status fixreleased

[Justin Marshall (jdmarshall90)]

Cool, thank you very much!

[Justin Marshall (jdmarshall90)]

3.39.1 fixed that problem, but I am now noticing something else: in the array of books in the response, I don't get a cover or thumbnail property in the JSON. I would expect to get this back, because the response for the "/ajax/book/" endpoint gives these.

I could always programmatically generate this URL client side based on the book ids in the response, but that would feel ... hacky. I would rather the endpoint output match "/ajax/book/"'s output.

Are these missing URLs intentional, or is this a defect?

Thanks

[Kovid Goyal (kovid)]

Nothing hacky about it, programmatically generating the URLs is what
calibre itself does in its web-interface.

[Justin Marshall (jdmarshall90)]

Meh, it just makes the client even more tightly coupled to the backend API. Not a huge deal, but something I'd like to avoid.

I've already implemented it though.