Cannot connect to BNRV1300 B&N GL4+

[David Suarez]

Calibre is unable to connect to Barnes & Noble Nook GlowLight4 Plus - BNRV1300. The NOOK folder is present under the 'Internal shared storage' drive. I am able to view the folder in Windows Explorer (Windows 10) and can create folders under the NOOK folder and copy files there. The device does not allow me to create folders or files under the 'Internal shared storage' location. Details of error message is below. Nook is running Platform version 8.1.7.31(E), App version 6.6.5.9 and shop version 6.6.5.3.

calibre, version 7.12.0
ERROR: Error: Error communicating with device

Failed to create the folder: NOOK in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

Traceback (most recent call last):
  File "calibre\devices\mtp\windows\driver.py", line 412, in create_folder
wpd.WPDError: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "calibre\gui2\device.py", line 111, in run
  File "calibre\gui2\device.py", line 565, in _books
  File "calibre\devices\mtp\driver.py", line 268, in books
  File "calibre\devices\mtp\driver.py", line 241, in get_driveinfo
  File "calibre\devices\mtp\driver.py", line 231, in _update_drive_info
  File "calibre\devices\mtp\driver.py", line 200, in put_calibre_file
  File "calibre\devices\mtp\driver.py", line 453, in ensure_parent
  File "calibre\devices\mtp\windows\driver.py", line 37, in check_thread
  File "calibre\devices\mtp\windows\driver.py", line 414, in create_folder
OSError: Failed to create the folder: NOOK in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

[Kovid Goyal (kovid)]

calibre works fine with the GL4. I dont have this device to test with
myself but several users have reported it works. From that error message
for some reason the NOOK folder in the internal storage is not being
detected. Try resetting the device, that should fix it.

[David Suarez (davids569)]

I have reset the device as well as uninstalled and reinstalled Calibre, no change. From the error messages, it appears that the code is trying to create the NOOK folder, but it is already present. And the permissions on the device do not allow folders to be created under 'Internal shared storage'. The device updated itself upon setup the first time, so I suspect that something has changed on the device that is causing it to now not connect correctly.

[Kovid Goyal (kovid)]

Yes, the attempt to make the folder happens only if the code thinks the
folder doesnt exist. As I dont have the device I cant really help you
further, but if you are willing to run calibre from source it should be
fairly easy to narrow down the cause of that.

Running from source is easy, see:
https://manual.calibre-ebook.com/develop.html

[Keith Agar (kwa61)]

I have the exact same error message being reported with my Barnes & Noble Nook GlowLight4 Plus - BNRV1300.
Using Calibre Release: 7.15.
Nook is running Platform version 8.1.7.38(E), App version 6.6.7.11 and shop version 6.6.7.2.
I can see in Windows Explorer "This PC\BNRV1300\Internal shared storage\" and the folders "NOOK", "Android", and ".adobe-digital-editions" under the internal shared storage.

[Rodrigo Pantoja (rpantojam)]

Same error here Nook BNRV1300, Platform 8.1.7.50(E), App 6.7.1.15

calibre, version 7.19.0
ERROR: Error: Error communicating with device

Failed to create the folder: NOOK in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

Traceback (most recent call last):
  File "calibre\devices\mtp\windows\driver.py", line 412, in create_folder
wpd.WPDError: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "calibre\gui2\device.py", line 112, in run
  File "calibre\gui2\device.py", line 566, in _books
  File "calibre\devices\mtp\driver.py", line 267, in books
  File "calibre\devices\mtp\driver.py", line 240, in get_driveinfo
  File "calibre\devices\mtp\driver.py", line 230, in _update_drive_info
  File "calibre\devices\mtp\driver.py", line 199, in put_calibre_file
  File "calibre\devices\mtp\driver.py", line 452, in ensure_parent
  File "calibre\devices\mtp\windows\driver.py", line 37, in check_thread
  File "calibre\devices\mtp\windows\driver.py", line 414, in create_folder
OSError: Failed to create the folder: NOOK in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

Also I can see the eReader in explorer and folder NOOK does exist

[Tempest (tempythecleric)]

Any movement or solutions for this because I am also having the same issue.

calibre, version 7.24.0
ERROR: Error: Error communicating with device

Failed to create the folder: NOOK in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

Traceback (most recent call last):
  File "calibre\devices\mtp\windows\driver.py", line 412, in create_folder
wpd.WPDError: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "calibre\gui2\device.py", line 112, in run
  File "calibre\gui2\device.py", line 566, in _books
  File "calibre\devices\mtp\driver.py", line 267, in books
  File "calibre\devices\mtp\driver.py", line 240, in get_driveinfo
  File "calibre\devices\mtp\driver.py", line 230, in _update_drive_info
  File "calibre\devices\mtp\driver.py", line 199, in put_calibre_file
  File "calibre\devices\mtp\driver.py", line 452, in ensure_parent
  File "calibre\devices\mtp\windows\driver.py", line 37, in check_thread
  File "calibre\devices\mtp\windows\driver.py", line 414, in create_folder
OSError: Failed to create the folder: NOOK in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:619:Failed to create folder:[hr=0x80004005 wCode=0] Unspecified error

[Embryonic (alphatjh-g)]

Hey Kovid, Thanks for an awesome application.

I Also have this issue on my Windows Machine, but it works on my Linux Machine.

The BNRV1300 already HAS a NOOK folder on the device by default, so I think this issue is due to the windows version trying to create the folder and getting an error, and possibly not handling it correctly.

I Tried deleting the NOOK Folder and letting Calibre create it, but the nook appears to re-create the folder upon being deleted.

The default filesystem is laid out below:

BNRV1300
-Internal Storage
--.adobe-digital-editions
--Android
--NOOK

Hope that helps!

[Kovid Goyal (kovid)]

See comment #4

[Kovid Goyal (kovid)]

Also try running the following command in a command prompt with your Nook connected and posting the output

```
calibre-debug -c "from calibre.devices.mtp.driver import *; main()"
```

That will dump the filesystem metadata which might tell us more about why the folder is not being seen.

[Tempest (tempythecleric)]

I haven't done the step in comment #4 yet, but here is the output from the command prompt in comment #10.

```
C:\Windows\System32>calibre-debug -c "from calibre.devices.mtp.driver import *; main()"
MTP: Loading filesystem metadata...
MTP: Filesystem metadata loaded in 0.0535738 seconds (0 objects)
+ Internal shared storage [id:s10001 0 children modified=1969/12/31 16:00]
Prefix for main mem: Books
Traceback (most recent call last):
  File "calibre\devices\mtp\windows\driver.py", line 412, in create_folder
wpd.WPDError: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:374:Failed to get properties for object:[hr=0x80042009 wCode=7689] IDispatch error #7689

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "runpy.py", line 198, in _run_module_as_main
  File "runpy.py", line 88, in _run_code
  File "site.py", line 83, in <module>
  File "site.py", line 78, in main
  File "site.py", line 50, in run_entry_point
  File "calibre\debug.py", line 245, in main
  File "<string>", line 1, in <module>
  File "calibre\devices\mtp\driver.py", line 711, in main
  File "calibre\devices\mtp\windows\driver.py", line 37, in check_thread
  File "calibre\devices\mtp\windows\driver.py", line 414, in create_folder
OSError: Failed to create the folder: developing-mtp-driver in ('Internal shared storage',) with error: C:\r\src\src\calibre\devices\mtp\windows\content_enumeration.cpp:374:Failed to get properties for object:[hr=0x80042009 wCode=7689] IDispatch error #7689
```

[Kovid Goyal (kovid)]

Internal shared storage [id:s10001 0 children modified=1969/12/31 16:00]

shows that its not reading anything inside the internal storage (see 0 children)

On a properly working device you would get something like

Internal Storage [id:s10001 11 children modified=1970/01/01 05:30]

It indicates the windows WPD EnumObjects() function is failing to return any children for the internal storage object or getting the properties for the children is failing for some reason.

[Kovid Goyal (kovid)]

In filesystem_cache.py print out all_storage and entries in the constructor of FilesystemCache that should confirm if the driver is returning entries or not.

[Tempest (tempythecleric)]

I am not sure how to do that, so either someone else will have too, or I will need to take some time to learn what that is / how to do that.

[Kovid Goyal (kovid)]

You setup the calibre dev env as described in in: https://manual.calibre-ebook.com/develop.html

Then apply the following patch and run the above command again:

diff --git a/src/calibre/devices/mtp/filesystem_cache.py b/src/calibre/devices/mtp/filesystem_cache.py
index b9a48579cc..c1844dd9ac 100644
--- a/src/calibre/devices/mtp/filesystem_cache.py
+++ b/src/calibre/devices/mtp/filesystem_cache.py
@@ -236,6 +236,10 @@ def mtp_id_path(self):
 class FilesystemCache:

     def __init__(self, all_storage, entries):
+ for x in all_storage:
+ print(x)
+ for x in entries:
+ print(x)
         self.entries = []
         self.id_maps = defaultdict(dict)
         self.all_storage_ids = tuple(x['id'] for x in all_storage)

[Rodrigo Pantoja (rpantojam)]

Hi, I've just setup the development environment and run

calibre-debug -c "from calibre.devices.mtp.driver import *; main()"

with a similar but somewhat different result than Tempest:

C:\Program Files\Calibre2>calibre-debug.exe -c "from calibre.devices.mtp.driver import *; main()"
MTP: Loading filesystem metadata...
MTP: Filesystem metadata loaded in 0.019624 seconds (0 objects)
+ Internal shared storage [id:s10001 0 children modified=1969/12/31 21:00]
{'device_version': '1.0',
 'friendly_name': '',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'BarnesAndNoble',
 'model_name': 'BNRV1300',
 'protocol': 'MTP: 1.00',
 'serial_number': '5043010011337008',
 'storage': [{'capacity': 29152509952,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 27541065728,
              'id': 's10001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'phone'}
Prefix for main mem: Books
Traceback (most recent call last):
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 730, in main
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 394, in list_folder_by_name
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 38, in check_thread
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 399, in list_mtp_folder_by_name
AttributeError: 'wpd.Device' object has no attribute 'list_folder_by_name'

Adding the changes to filesystem_cache.py to list all_storage and entries:

C:\Program Files\Calibre2>calibre-debug.exe -c "from calibre.devices.mtp.driver import *; main()"
MTP: Loading filesystem metadata...
{'id': 's10001', 'size': 29152509952, 'name': 'Internal shared storage', 'is_folder': True, 'can_delete': False, 'is_system': True}
MTP: Filesystem metadata loaded in 0.027566 seconds (0 objects)
+ Internal shared storage [id:s10001 0 children modified=1969/12/31 21:00]
{'device_version': '1.0',
 'friendly_name': '',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'BarnesAndNoble',
 'model_name': 'BNRV1300',
 'protocol': 'MTP: 1.00',
 'serial_number': '5043010011337008',
 'storage': [{'capacity': 29152509952,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 27541065728,
              'id': 's10001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'phone'}
Prefix for main mem: Books
Traceback (most recent call last):
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 730, in main
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 394, in list_folder_by_name
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 38, in check_thread
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 399, in list_mtp_folde...

[Kovid Goyal (kovid)]

Yes, that's because I have changed what main() does for unrelated
reasons, but the problem remains the same. EnumObjects() is not
returning any children for the Internal storage. As long as that's the
case there is no way calibre can work with this device.

[Rodrigo Pantoja (rpantojam)]

Thanks Kovid. I see what you mean. I'm an IT profesional and even though my knowledge of Python and C++ is superficial for the first and very rusty for the second, I'll try and investigate why the WPD API is failing on this device. It took me a while but I found the EnumObjects call in device_enumeration.cpp . I have an educated guess that maybe the WPD_DEVICE_OBJECT_ID constant is causing trouble on this particular device. I'm installing the Windows SDK right now, it seems one of the samples might have enough functionality to help me investigate further. I'll keep you posted

[Tempest (tempythecleric)]

Because all of this does mostly go over my head. If either of you do figure out a solution or find a workaround let me know please.

[Kovid Goyal (kovid)]

Yeah if you can figure out how to get enumobjects working with this device I can make the needed changes in the C++ code in the calibre driver.

[Tempest (tempythecleric)]

Just to add this, I set up the environment, added the patch and ran the command again and here is the output:

```
C:\>calibre-debug -c "from calibre.devices.mtp.driver import *; main()"
Hello, world!
MTP: Loading filesystem metadata...
{'id': 's10001', 'size': 29152509952, 'name': 'Internal shared storage', 'is_folder': True, 'can_delete': False, 'is_system': True}
MTP: Filesystem metadata loaded in 0.030947 seconds (0 objects)
+ Internal shared storage [id:s10001 0 children modified=1969/12/31 16:00]
{'device_version': '1.0',
 'friendly_name': '',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'BarnesAndNoble',
 'model_name': 'BNRV1300',
 'protocol': 'MTP: 1.00',
 'serial_number': '5043010007857000',
 'storage': [{'capacity': 29152509952,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 27454205952,
              'id': 's10001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'phone'}
Prefix for main mem: Books
Traceback (most recent call last):
  File "C:\Users\miral\calibre\src\calibre\devices\mtp\driver.py", line 731, in main
  File "C:\Users\miral\calibre\src\calibre\devices\mtp\driver.py", line 395, in list_folder_by_name
  File "C:\Users\miral\calibre\src\calibre\devices\mtp\windows\driver.py", line 38, in check_thread
  File "C:\Users\miral\calibre\src\calibre\devices\mtp\windows\driver.py", line 399, in list_mtp_folder_by_name
AttributeError: 'wpd.Device' object has no attribute 'list_folder_by_name'
```

[Tempest (tempythecleric)]

I also have no idea is anything from here will help, but here it is

[Rodrigo Pantoja (rpantojam)]

Kovid, setting up my PC for WPD development is becoming a PITA, mostly because my system disk is out of space. I will solve it eventually. But meanwhile I was looking at the WPD samples I mentioned earlier and also, prompted by Tempest's screen grab, at the WPD info tool. It seems to me that the sample/tools do a recursive enumeration, i.e. they first call EnumObjects with WPD_DEVICE_OBJECT_ID as ID and then they use the list obtained in this first call to keep calling EnumObjects with the ids obtained in the previous call and so on. From the test we run on Calibre we see that the enumeration returns at least one object:
{'id': 's10001', 'size': 29152509952, 'name': 'Internal shared storage', 'is_folder': True, 'can_delete': False, 'is_system': True}.

Could you compile a version of the system where you change device_enumeration.cpp and replace:
    hr = content->EnumObjects(0, WPD_DEVICE_OBJECT_ID, NULL, &objects);
for:
    hr = content->EnumObjects(0, "s10001", NULL, &objects);
just to check if this works with GL4+? If this does work I guess you can see the way to a fix.

[Tempest (tempythecleric)]

Rodrigo, let me know if there is a WPD test or screenshot you want me to take since I have Windows SKD and Dev tools up and running to try and diagnose it.

[Kovid Goyal (kovid)]

calibre also does a recursive enumeration and indeed does the
enumeration with the storage id, see the file content_enumeration.cpp
get_files_and_folders() function

WPD_DEVICE_OBJECT_ID is just used in device_enumeration.cpp to get the
storage ids, each storage id is then recursively listed.

If it didnt do that it wouldnt work with any MTP device. The question is
basically why EnumObjects() doesnt work when called with the storage id
of the Internal Storage

[Rodrigo Pantoja (rpantojam)]

Understood. I'll keep quiet, finish up setting up my pc and do the tests. I'll get back once I've been able to have an answer to Kovid's question "why EnumObjects() doesnt work when called with the storage id
of the Internal Storage [?]"

[Rodrigo Pantoja (rpantojam)]

I installed Visual Studio and loaded a sample project that does basic operations on portable devices using the WPD API. It's like a lightweight, text based WPD Info Tool (the one Tempest used). Just running it on the Nook worked just fine, it showed all the folders and by debugging it I found that the answer to Kovid's question is: it does work when calling EnumObjects with id "s10001" and the call actually returns the objects inside (see attached screen grab). Unfortunately the problem must lie somewhere else within calibre. Only hint I have is that the WPD Info Tool does show an error "HRESULT = 0x80042009" when opening Nook and it doesn't when opening other readers, e.g. Kobo libra colour nor Kindle Oasis. It still does show all the information, but throws this error when opening the device. The description of the error code says "Invalid ObjectHandle, Indicates that an object handle does not refer to an actual object that is present on the device. The application should enumerate the storages again." Could it be that something similar is happening inside calibre and that you need to renumerate? I think this is as far as I can get without actually debugging calibre itself, or at least the driver. Which I'm willing to do if somebody tells me how to set it up. Of course I'm available to help in any other way you can suggest.

[Kovid Goyal (kovid)]

Try it with the attached wpd.pyd, I have change the code to actually fail when calling Next() on the result if EnumObjects() returns anything other than S_OK or S_FALSE. That should at least tell us exactly where it is failing.

[Rodrigo Pantoja (rpantojam)]

Done, changed wtp.pyd in <calibre_installation>\app\bin folder. It looks the same to me:
C:\Program Files\Calibre2>calibre-debug.exe -c "from calibre.devices.mtp.driver import *; main()"
MTP: Loading filesystem metadata...
MTP: Filesystem metadata loaded in 0.0384645 seconds (0 objects)
+ Internal shared storage [id:s10001 0 children modified=1969/12/31 21:00]
{'device_version': '1.0',
 'friendly_name': '',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'BarnesAndNoble',
 'model_name': 'BNRV1300',
 'protocol': 'MTP: 1.00',
 'serial_number': '5043010011337008',
 'storage': [{'capacity': 29152509952,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 27540688896,
              'id': 's10001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'phone'}
Prefix for main mem: Books
Traceback (most recent call last):
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 730, in main
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 394, in list_folder_by_name
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 38, in check_thread
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 401, in list_mtp_folder_by_name
FileNotFoundError: Could not find folder named: Books in ('Internal shared storage',)

[Kovid Goyal (kovid)]

That's rather odd, just to do a sanity check, here is a version that explicitly prints out the number of children returned by EnumObjects. Can you run with this and see what it prints out. Unfortunately getting calibre to build on windows from source is highly non-trivial. If you have a linux machine you can easily cross compile for windows though. Anyway, this is the patch I applied:

diff --git a/src/calibre/devices/mtp/windows/content_enumeration.cpp b/src/calibre/devices/mtp/windows/content_enumeration.cpp
index db149d4f6c..6bb51f5021 100644
--- a/src/calibre/devices/mtp/windows/content_enumeration.cpp
+++ b/src/calibre/devices/mtp/windows/content_enumeration.cpp
@@ -569,6 +569,8 @@ get_files_and_folders(unsigned int level, IPortableDevice *device, CComPtr<IPort
         fwprintf(stderr, L"Ignoring failure of EnumObjects() at level %u\n", level); fflush(stderr);
         return true;
     }
+ DWORD num; object_ids->GetCount(&num);
+ fprintf(stderr, "EnumObjects() return %lu children for folder %ls\n", num, parent_id);

     if (bulk_properties != NULL) {
                if (!bulk_get_filesystem(level, device, bulk_properties, object_ids, callback, ans, subfolders.ptr())) return false;

This patch should confirm if the issue is in EnumObjects or somewhere later.

[Rodrigo Pantoja (rpantojam)]

Interesting:
C:\Program Files\Calibre2>calibre-debug.exe -c "from calibre.devices.mtp.driver import *; main()"
MTP: Loading filesystem metadata...
EnumObjects() return 13 children for folder s10001
MTP: Filesystem metadata loaded in 0.0391161 seconds (0 objects)
+ Internal shared storage [id:s10001 0 children modified=1969/12/31 21:00]
{'device_version': '1.0',
 'friendly_name': '',
 'has_bulk_properties': True,
 'has_storage': True,
 'manufacturer_name': 'BarnesAndNoble',
 'model_name': 'BNRV1300',
 'protocol': 'MTP: 1.00',
 'serial_number': '5043010011337008',
 'storage': [{'capacity': 29152509952,
              'capacity_objects': 0,
              'description': 'Internal shared storage',
              'filesystem': 'Generic hierarchical',
              'free_objects': 1073741824,
              'free_space': 27540688896,
              'id': 's10001',
              'name': 'Internal shared storage',
              'rw': True,
              'type': 'fixed_ram'}],
 'type': 'phone'}
Prefix for main mem: Books
Traceback (most recent call last):
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 730, in main
  File "C:\calsrc\calibre\src\calibre\devices\mtp\driver.py", line 394, in list_folder_by_name
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 38, in check_thread
  File "C:\calsrc\calibre\src\calibre\devices\mtp\windows\driver.py", line 401, in list_mtp_folder_by_name
FileNotFoundError: Could not find folder named: Books in ('Internal shared storage',)

[Rodrigo Pantoja (rpantojam)]

As for the Linux environment, not my cup of tea. Maybe I could try it on a VM. Lets see how much we can accomplish working this way.

[Rodrigo Pantoja (rpantojam)]

I see that in get_files_and_folders right after the line you added to print the number of children, you check if bulk_properties is null and either call bulk_get_filesystem or single_get_filesystem. I ran the sample app again trying to do a bulk read since it seems to be supported by the call to queryinterface. BUT the callback fails on method OnEnd with the same error I saw in the WPD info tool 0x80042009

[Rodrigo Pantoja (rpantojam)]

Hi, I found the problem. It is quite late I'm tired and want to go to sleep. I'll try to explain it better tomorrow. Summary, the storage has many children that give an error when reading their properties, be it in bulk or normally. The problem is that if you do it in bulk and mix the ones who throw an error with the ones that work correctly, everything fails, even the ones that you would be able to read if you queried them separately.

[Kovid Goyal (kovid)]

But that's pretty absurd, if getting properties for enumerable objects
fail how is the application supposed to know which of these to ignore
and which to abort on? Is there some pattern to the ones that fail? I have attached a
version of the driver that does not use bulk operations on this device (as identified by manufacturer and model names) and also that correctly reports failures and aborts in OnEnd on bulk operations. It also retries failed bulk gets with single gets and ignores objects for which the single get fails.

[Kovid Goyal (kovid)]

And here is a version that reports symbolic error names for known MTP subsystem errors, should make diagnosis a bit easier.

[Rodrigo Pantoja (rpantojam)]

Yes, it is absurd. I looks like a lazy/buggy implementation of the protocol and a testament to how robust windows drivers are (and surely Linux ones too). If we hadn't had problems with Calibre we'd never had noticed anything wrong with the device and maybe that is what B&N QA experienced too.
  BTW, I did a very thorough testing both on the WPD Info tool and with the WpdApiSample example Visual Studio project. I was quite confident in what I said, but I did test again today just in case I was too tired yesterday and had misinterpreted something.
  I can confirm these results:
  EnumObjects returns 13 objects from id "s10001", only "o6", "o13" and "o14" work properly on WPD info tool reading their properties, and these three actually are the three folders visible in Windows Explorer: "Android", "NOOK" and ".adobe-digital-editions". All other objects fail on this tool with error 0x80042009.
  Using WpdApiSample things have become much weirder. Up until yesterday night I was able to replicate what I saw on the WPD info tool. However, I thought the problem might be related to what properties you're trying to read because it seems that wpd info tool tries to read them all. I tested only reading the name (WPD_OBJECT_NAME) and everything worked well, both individually and in bulk with all objects. Cool, it seems to be what I thought. I started adding the other properties one by one WPD_OBJECT_CONTENT_TYPE, WPD_OBJECT_FORMAT and WPD_OBJECT_PERSISTENT_UNIQUE_ID since this program only uses these four. Guess what? Now it works!! I removed the query for all properties but the name, and then added one by one the query for the others until I was asking for the same four the program was using originally and it doesn't fail anymore. I was so surprised that I downloaded the sample code again, and uncompressed it to a new folder, built this solution from zero, and it also works!!!. I even rebooted and it still works fine. I don't know what to make of it. Now with this program everything works correctly and I can't replicate the problems. I have absolutely no idea what I could've changed. I tried both calibre and the wpd tool and they still have the same issues. I'll keep thinking what might've changed in the environment, but for the time being we've lost a tool. Of course if I find what caused the change I'll let you know to see if it is applicable to calibre.
  As for testing with the last wpd.pyd this is what I got:
C:\Program Files\Calibre2>calibre-debug.exe -c "from calibre.devices.mtp.driver import *; main()"
MTP: Loading filesystem metadata...
Ignoring object with id: o1 because getting its properties failed.
  That's all no, more output. When running calibre.exe with this wpd.pyd, it loads and then exits abruptly.

[Kovid Goyal (kovid)]

Apologies, I had a stupid typo in the error handling path of get_properties() failing, this should fix it.

[Kovid Goyal (kovid)]

And this version also prints out the HRESULT on get_properties failure. Which might give us some insight.

[Rodrigo Pantoja (rpantojam)]

This looks very promising. Attached is the output of calibre-debug.exe -c "from calibre.devices.mtp.driver import *; main()" with the latest wpd.pyd

[Kovid Goyal (kovid)]

So is it working with normal operations in calibre?

[Rodrigo Pantoja (rpantojam)]

Sorry, forgot about that. Disregarding all the error popups it works. First it connects to the device. It lists the device content. It allows sending content. The content shows up in the device. The content can be open and read in the device. I was able to delete content. I can open books from the device on the calibre book reader. The book covers carrousel is not updated when listing the device contents, it keeps showing the main library covers. Please let me know if you want me to do any other specific test.

[Kovid Goyal (kovid)]

What error popups? I have added a bunch of prints but no new error
popups.

[Rodrigo Pantoja (rpantojam)]

I mean this error messages that appear on calibre connecting to Nook. I don't know if I misused the term "error popup". I get 10 of these and if I OK all of them, then calibre works as I explained on comment 42

[Kovid Goyal (kovid)]

Those are likely coming from create_folder() or put_file() calls can you post the
full traceback.

[Kovid Goyal (kovid)]

You get the full traceback by clicking get details.

[Rodrigo Pantoja (rpantojam)]

Attached is the traceback from one of the windows, they're all 10 the same

[Kovid Goyal (kovid)]

Here is another version to try

[Rodrigo Pantoja (rpantojam)]

The behavior of Calibre using this version looks normal. I mean no extraneous messages, and Nook works well, as described on note 42.

[Kovid Goyal (kovid)]

OK, marking this as fixed then.

[Rodrigo Pantoja (rpantojam)]

Thanks Kovid, pleasure working with you and many many thanks for your great application and the effort maintaining and enhancing Calibre.

[Kovid Goyal (kovid)]

You are welcome and thanks for helping with the debugging and testing,
this particular issue really needed someone with the hardware to do the
work of nailing down the root cause.

I really wish B&N would fix their MTP firmware though.

[Embryonic (alphatjh-g)]

Thank you Kovid and Rodrigo for working through this, Look forward to the fruits of your labor! You guys rock.

[Rodrigo Pantoja (rpantojam)]

Kovid said "I really wish B&N would fix their MTP firmware though." Totally agree, this situation certainly looks a lot more like a bug than a feature.

[Tempest (tempythecleric)]

Sorry I haven't been active. Does this mean we have a solution? Will there be an update to Calibre or do we need to do something manually to get it functioning now. Thank you so much for all your work!

[Kovid Goyal (kovid)]

Simply wait for the next release or try a preview release:
https://www.mobileread.com/forums/showthread.php?t=359648