Programs not in guide

[Arisian]

Downloading Schedule Direct via XMLTV does not properly update guide. Specifically see the snapshot attached and the XML shown. This is what *should* have been shown in the time in question.

  <programme start="20080914183000 -0400" stop="20080914203000 -0400" channel="I10741.labs.zap2it.com">
    <title lang="en">The Brothers Grimm</title>
    <desc lang="en">Itinerant con men (Matt Damon, Heath Ledger) become caught in a real fairy tale after several maidens disappear in an enchanted forest.</desc>
    <credits>
      <director>Terry Gilliam</director>
      <actor>Matt Damon</actor>
      <actor>Heath Ledger</actor>
      <actor>Peter Stormare</actor>
      <actor>Lena Headey</actor>
      <actor>Jonathan Pryce</actor>
      <actor>Monica Bellucci</actor>
      <actor>Mackenzie Crook</actor>
      <actor>Roger Ashton-Griffiths</actor>
      <actor>Harry Gilliam</actor>
      <actor>Mirosláv Táborský</actor>
      <actor>Jan Unger</actor>
      <actor>Richard Ridings</actor>
      <writer>Ehren Kruger</writer>
      <producer>Daniel Bobker</producer>
      <producer>Chuck Roven</producer>
    </credits>
    <date>2005</date>
    <category lang="en">Fantasy</category>
    <category lang="en">Adventure</category>
    <length units="minutes">118</length>
    <episode-num system="dd_progid">MV00163615.0000</episode-num>
    <rating system="advisory">
      <value>Adult Situations</value>
    </rating>
    <rating system="advisory">
      <value>Violence</value>
    </rating>
    <rating system="MPAA">
      <value>PG-13</value>
    </rating>
    <star-rating>
      <value>2/4</value>
    </star-rating>
  </programme>

There are at least a dozen or more examples in my current guide of blank spaces that aren't being filled in no matter how many times I download the XML (I have the specific XML cached so it reloads the same file each time).

[Andy Balaam (mail-artificialworlds)]

I can confirm that FreeGuide is unable to import the XMLTV snippet shown above. It complains about an invalid unicode character.

If I replace "Mirosláv Táborský" with "Miroslav Taborsky" (no accents) it imports fine.

I wonder whether Schedules Direct is providing invalid characters or FreeGuide is interpreting them wrong. Please could you contact the XMLTV developers to ask whether these characters are supposed to be unicode, and if so whether they are incorrect?

Thanks, Andy

[Arisian (mmclagan)]

Unfortunately, it doesn't make a difference whether or not the accents are in place. Another gap in the guide is visible in the second snapshot. The snippet of XML that should have been imported and shown is:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">

<tv source-info-url="http://www.schedulesdirect.org/" source-info-name="Schedules Direct" generator-info-name="XMLTV/$Id: tv_grab_na_dd.in,v 1.69 2007/11/15 05:19:21 rmeden Exp $" generator-info-url="http://www.xmltv.org/">
  <channel id="I16318.labs.zap2it.com">
    <display-name>617 SHCASEP</display-name>
    <display-name>617 SHCASEP 0005993:X</display-name>
    <display-name>617</display-name>
    <display-name>SHCASEP</display-name>
    <display-name>Showcase Television Inc. (Pacific)</display-name>
    <display-name>Canadian Satellite</display-name>
  </channel>
  <programme start="20080914190000 -0400" stop="20080914214500 -0400" channel="I16318.labs.zap2it.com">
    <title lang="en">The Terminal</title>
    <desc lang="en">Unauthorized to enter the United States, an Eastern European (Tom Hanks) befriends a flight attendant (Catherine Zeta-Jones) while living in a New York airport.</desc>
    <credits>
      <director>Steven Spielberg</director>
      <actor>Tom Hanks</actor>
      <actor>Catherine Zeta-Jones</actor>
      <actor>Stanley Tucci</actor>
      <actor>Chi McBride</actor>
      <actor>Diego Luna</actor>
      <actor>Barry Shabaka Henley</actor>
      <actor>Kumar Pallana</actor>
      <actor>Eddie Jones</actor>
      <actor>Jude Ciccolella</actor>
      <actor>Corey Reynolds</actor>
      <actor>Guillermo Díaz</actor>
      <actor>Rini Bell</actor>
      <actor>Stephen Mendel</actor>
      <actor>Valera Nikolaev</actor>
      <actor>Michael Nouri</actor>
      <actor>Zoe Saldana</actor>
      <writer>Sacha Gervasi</writer>
      <writer>Jeff Nathanson</writer>
      <producer>Laurie MacDonald</producer>
      <producer>Walter F. Parkes</producer>
      <producer>Steven Spielberg</producer>
    </credits>
    <date>2004</date>
    <category lang="en">Comedy-drama</category>
    <length units="minutes">128</length>
    <episode-num system="dd_progid">MV00149162.0000</episode-num>
    <audio>
      <stereo>stereo</stereo>
    </audio>
    <rating system="advisory">
      <value>Adult Situations</value>
    </rating>
    <rating system="advisory">
      <value>Language</value>
    </rating>
    <rating system="MPAA">
      <value>PG-13</value>
    </rating>
    <star-rating>
      <value>3/4</value>
    </star-rating>
  </programme>
</tv>

Also, I modified the XML from the original report, changed 147 á and 5 ý in the file and re-imported it. It didn't replace the bogus program.

[Andy Balaam (mail-artificialworlds)]

I imported the above snippet and it appeared in FreeGuide fine.

Could you try deleting all the .ser files in the working directory (.freeguide) and try importing just this snippet?

Does "The Terminal" overlap with another programme in the listings coming from XMLTV? This could cause it to be removed when the other programme is imported.

[Arisian (mmclagan)]

I've no doubt that the segment is importable. What isn't happening is that the program is not updating properly, removing the conflicting programs and inserting the newly imported one.

On 2008-09-02, SD served up:

  <programme start="20080914190000 -0400" stop="20080914211500 -0400" channel="I16318.labs.zap2it.com">
    <title lang="en">The Terminal</title>
    <desc lang="en">Unauthorized to enter the United States, an Eastern European (Tom Hanks) befriends a flight attendant (Catherine Zeta-Jones) while living in a New York airport.</desc>
    <credits>
      <director>Steven Spielberg</director>
      <actor>Tom Hanks</actor>
      <actor>Catherine Zeta-Jones</actor>
      <actor>Stanley Tucci</actor>
      <actor>Chi McBride</actor>
      <actor>Diego Luna</actor>
      <actor>Barry Shabaka Henley</actor>
      <actor>Kumar Pallana</actor>
      <actor>Eddie Jones</actor>
      <actor>Jude Ciccolella</actor>
      <actor>Corey Reynolds</actor>
      <actor>Guillermo Díaz</actor>
      <actor>Rini Bell</actor>
      <actor>Stephen Mendel</actor>
      <actor>Valera Nikolaev</actor>
      <actor>Michael Nouri</actor>
      <actor>Zoe Saldana</actor>
      <writer>Sacha Gervasi</writer>
      <writer>Jeff Nathanson</writer>
      <producer>Laurie MacDonald</producer>
      <producer>Walter F. Parkes</producer>
      <producer>Steven Spielberg</producer>
    </credits>
    <date>2004</date>
    <category lang="en">Comedy-drama</category>
    <length units="minutes">128</length>
    <episode-num system="dd_progid">MV00149162.0000</episode-num>
    <audio>
      <stereo>stereo</stereo>
    </audio>
    <rating system="advisory">
      <value>Adult Situations</value>
    </rating>
    <rating system="advisory">
      <value>Language</value>
    </rating>
    <rating system="MPAA">
      <value>PG-13</value>
    </rating>
    <star-rating>
      <value>3/4</value>
    </star-rating>
  </programme>

On 2008-09-03, SD added:
  <programme start="20080914211500 -0400" stop="20080914213000 -0400" channel="I16318.labs.zap2it.com">
    <title lang="en">To Be Announced</title>
    <episode-num system="dd_progid">SH00019168.0000</episode-num>
    <previously-shown />
  </programme>
  <programme start="20080914213000 -0400" stop="20080914220000 -0400" channel="I16318.labs.zap2it.com">
    <title lang="en">Billable Hours</title>
    <sub-title lang="en">The Jewish Holiday</sub-title>
    <desc lang="en">The agnostic but half-Jewish Sam begins wearing a yarmulke to work to cover his bald spot.</desc>
    <date>20060430</date>
    <category lang="en">Comedy</category>
    <category lang="en">Limited Series</category>
    <episode-num system="dd_progid">EP00818735.0003</episode-num>
    <episode-num system="onscreen">1003</episode-num>
    <audio>
      <stereo>stereo</stereo>
    </audio>
    <previously-shown start="20060430000000" />
    <subtitles type="teletext" />
  </programme>

On 2008-09-13, SD served up:
  <programme start="20080914190000 -0400" stop="20080914214500 -0400" channel="I16318.labs.zap2it.com">
    <title lang="en">The Terminal</title>
    <desc lang="en">Unauthorized to enter the United States, an Eastern European (Tom Hanks) befriend...

[Andy Balaam (mail-artificialworlds)]

Thanks for this excellent writeup. I will create an automated test case for this situation and try to reproduce it, and get a fix in as soon as I get time.

[Arisian (mmclagan)]

I tried removing all the SER files in use prior to using 10.9, it has not made any significant difference. The guide continues to miss entries, sometimes in blocks.

The most recent missing programs all seem to come about when the OLD schedule contains two or more programs that are replaced by one -- the updated schedule has a 2 hour program that is replacing 2 one hour programs. Today in particular the biography channel (BIO 266 on DirecTV) replaced it's regular programming with a Paul Newman retrospective and Discovery (DSC 278) replaced it's regular programs with a 2 hour special called the "Ten Commandments of the Mafia". Both these updates left a 1 hour block with no program and the old one hour block -- the empty block is at the start of the timeslot, the old program that wasn't removed starts in the 2nd hour and ends at the same time as the new program should have.

Old programs: 00:00-01:00, 01:00-02:00, new programs: 00:00-02:00. Guide shows blank 00:00-01:00, old program 01:00-02:00.

If you need these or other similar events, let me know.

[Andy Balaam (mail-artificialworlds)]

Fixed in svn revision 1714.

Thanks again for the clear and detailed bug report. I was able to write an automated test for this bug and I used that to help me track down and fix the problem.

[Andy Balaam (mail-artificialworlds)]

Fixed in release 0.10.10.