If you’re using MythTV 0.28 on Ubuntu 16.04 …

… you’ll want to know about this bug.  Put the following string in the end of your /etc/mysql/conf.d/mythtv.cnf ;

sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

You may also want to try;

sql_mode=NO_ENGINE_SUBSTITUTION

Fixed?

Recordings Purge Script for MythTV

Edit 25th Feb:  Discovered a bug in the logic, apparently UNION order isn’t guaranteed.  Reworked the script so it does the right thing now.

I’ve now converted my prior HTPC setup over to using an HDHomeRun and MythTV on my Microserver.  The former MediaPortal box now runs exclusively OpenElec XBMC.  Anyway, MythTV has a proper database backend and a set of Python/Perl bindings, which is great for customization.

I have a pretty specific want for my recording schedules, which ARGUS (on MediaPortal) wasn’t able to do, and which MythTV is not quite able to do.  Namely, I want to record a certain number of a specific series, and then only record more of them if they get watched.  The oldest watched episode should be discarded to make room, so you always maintain N episodes that are unwatched, but you don’t record new episodes if you already have N episodes unwatched.

MythTV’s native “Expire old and record new” functionality does not do this (it will keep recording episodes to maintain N new episodes, overwriting older episodes even if unwatched with new ones).  The “Auto-expire” functionality won’t do this (it will only delete episodes if running out of disk space).

Enter the following script.  Yes, the SQL query is freakin’ huge.

#!/usr/bin/python -W ignore::DeprecationWarning
#
# Purges watched episodes from schedules so that new episodes can be
# recorded
#
# Only purges the oldest watched recording, and only if the schedule
# wouldn't record because it's reached the maximum episodes
#
# I'm terrible at Python.

from MythTV import MythDB, Recorded
import sys
import MySQLdb

db = MythDB()

cur = db.cursor()

# Fetch all candidate recording ids
cur.execute("""
SELECT
  candidates.recordid
FROM
  (
    SELECT
      record.recordid,
      record.title,
      record.maxepisodes,
      COUNT(recorded.basename) AS count
    FROM
      record,
      recorded
    WHERE
      record.autoexpire=1
      AND record.inactive=0
      AND record.maxnewest=0
      AND record.recordid = recorded.recordid
      AND NOT recorded.recgroup = 'LiveTV'
      AND NOT recorded.recgroup = 'Deleted'
      AND recorded.preserve = 0
      AND recorded.autoexpire = 1
    GROUP BY
      recorded.recordid
  ) AS candidates
WHERE
  candidates.maxepisodes <= candidates.count
""")

for record in cur.fetchall() :
        cur2 = db.cursor();
        cur2.execute("""
                SELECT
                  e.title,
                  e.subtitle,
                  e.basename,
                  e.starttime,
                  e.deletepriority
                FROM
                (
                  (
                  -- Subselect fully watched episodes
                  SELECT
                    b.title,
                    b.subtitle,
                    b.basename,
                    b.starttime,
                    1 AS deletepriority
                  FROM
                    recorded AS b
                  WHERE
                    b.recordid = ?
                    AND b.preserve=0
                    AND b.autoexpire=1
                    AND b.watched=1
                    AND NOT b.recgroup = 'LiveTV'
                    AND NOT b.recgroup = 'Deleted'
                  -- End of watched episodes
                  ) UNION (
                  -- Subselect bookmarked episodes
                  SELECT
                    c.title,
                    c.subtitle,
                    c.basename,
                    c.starttime,
                    2 AS deletepriority
                  FROM
                    recorded as c,
                    recordedmarkup as d
                  WHERE
                    c.recordid = ?
                    AND c.preserve=0
                    AND c.autoexpire=1
                    AND c.watched=0
                    AND NOT c.recgroup = 'LiveTV'
                    AND NOT c.recgroup = 'Deleted'
                    AND d.chanid = c.chanid
                    AND d.starttime = c.starttime
                    AND d.type = 2
                  -- End of bookmarked episodes
                  )
                ) AS e
                ORDER BY
                  e.deletepriority,
                  e.starttime
                LIMIT 1
        """, (record[0], record[0]))
        for row in cur2.fetchall() :
                print row[0] + " : " + row[1] + " (" + row[2] + ") [" + str(row[3]) + "]"
                recs = list(db.searchRecorded(basename=row[2]))
                if len(recs) == 0:
                        print '  error - could not find episode by basename!'
                        sys.exit(0)
                for rec in recs:
                        rec.delete()

I’ll break down what this does.

  • For each active autoexpirable recording schedule, see if the number of autoexpirable, nonpreserved, nondeleted recordings exceeds or matches the maximum number of allowable recordings.
  • For each of those schedules, find the oldest fully watched or bookmarked, nonpreserved, autoexpirable, nondeleted recording and delete it.

If you put this in crontab to run once a day, you’ll find that each of your schedules will do what I outlined above.

The great thing about this is that it allows you to set up recording schedules for anything (where you don’t care about episode order) that looks vaguely interesting, set an appropriate maximum (5 or so), and it’ll only keep recording them if they’re actually getting watched.

This is the functionality I want for children’s television, and now I’ve got it.

MythTV – Teething Problems

As discussed earlier, I’ve been rebuilding my Media Center with MythTV running on Fedora 15.  I’ve been having a number of … issues … with it.  Fortunately I’ve gotten a lot of them finally nailed down.

Tuner Issues

I had two Hauppauge HVR-2200 tuners.  One of them just flat out didn’t work, and the other worked, but the second tuner on it was frequently poor.  I’ve since pulled them both out and intend on putting them into my Windows main PC the next time I have it out to see if they work.

I’ve replaced them with a Leadtek WinFast DTV Dongle Gold and a Leadtek WinFast DTV2000DS.  The Gold is a USB key-format DVB-T tuner (single tuner), and the DTV2000DS is a PCI dual-tuner.

The Gold just worked.  And so did the DTV2000DS, for that matter.  The DTV2000DS interestingly enough is actually a dual USB tuner built onto a PCI to USB bridge.  Annoyingly though, the two tuners are completely indistinguishable to udev, resulting in an inability for udev to make distinct adapters for them.  A solution for this when I come up with one.

Anyway, I had a number of difficulties with them.  Primarily sometimes on powerup the second tuner doesn’t work, or it randomly stops working.  What I’ve done is to add this to /etc/rc.d/rc.local;

echo -n -1 > /sys/module/usbcore/parameters/autosuspend

This prevents the kernel suspending power to the USB ports, which apparently causes issues with these tuners.  Secondly, in a bid to make it work consistently, I also added the following to a new file /etc/modprobe.d/tda18271.conf;

options tda18271 cal=0

This disables radio calibration on module load for the tuners.  My suspicion is that radio calibration was throwing the tuners off when the modules are loading.  This may be bunk, but so far it works.

Lastly, I’ve also discovered a way to test the tuners from the command line.  I’m considering adding this to the bootup procedure so that I can verify that all three tuners are fully operational before letting MythTV use them – to prevent instances of a recording being made that’s garbage.


Fedora 16 Upgrade

In my bid to try and get all the tuners working properly (the other issue I had was the second tuner on the DTV2000DS suddenly stop working and not work again), I upgraded my MythTV box to Fedora 16.  This was actually pretty painless, except for one really big gotcha.

Fedora 15 Mythbackend runs as root.  Fedora 16 Mythbackend runs as mythtv.  This means that you won’t have write access to the spooler drive, and it basically breaks Myth.  Until I get around to fully fixing up the permissions, I’ve changed /lib/systemd/mythbackend.service to run as mythtv.

Sleep / Hibernation

I have a number of issues trying to get sleep/hibernate working.  If I use pm-suspend, the graphics card doesn’t restart.  If I use pm-hibernate, initially the box just simply locked up.  I determined the lockup was due to the tuner drivers, so I wrote a script to stop the Myth Backend service and unload the modules on hibernate.

But then I discovered that frequently on waking up some of the tuners wouldn’t work!  I haven’t got a solution for this yet, but I’m working on it.

All in all, it’s been a worthwhile exercise, but I’ve been getting a lot of grief from the TV tuners.  I’ve left it alone for this week, but in a week or so I’ll try and get hibernation working again.

BackupPC for MythTV

I was having some trouble getting BackupPC to back up my MythTV’s operating system to my Microserver.  I kept getting aborts all the time from broken pipes and such.

Got fatal error during xfer (aborted by signal=PIPE)
Anyway, the solution turns out to have been here.  Disable TCP segmentation offloading on the client end, and now it all works!

From WMC7 to MythTV…

After reading some more about what can be done with MythTV, I thought I’d take the plunge.  Previously, my Media Center was running Windows 7 Ultimate with Media Center.  I have two Hauppauge HVR-2200 PCI-Express dual tuners, and an nVidia 9500 GT graphics card with HDMI output.  I also have a huge pile of recorded TV in Windows Television (WTV) format, which was quite the obstacle because only Media Center can play that.

So, I decided to settle in with MythTV running on Fedora 15.  There’s a heap of build guides on the Web, follow them if you want to do similar.  I’ll just mostly talk about the issues I had.

Old Recorded TV

Firstly, I copied off all my previously spooled recorded TV in WTV format and using VideoRedo, remuxed them into plain MPEG2 Program Stream (.mpg) files, which MythTV (and nearly anything) can play.  At a later time I’ll transcode all the things I want to keep to H.264 in an MKV container, but for now I just want everything playable.  VideoRedo can remux a WTV pretty quickly, so I did all that on my main PC while I was messing with the HTPC.

HTPC Setup – OS Choices

I decided to go with Fedora 15 x64 instead of 16.  For one, 16 had just been released, and I was concerned there may be setup problems with it.  Fedora 15 is tried and true, and it should just “work”.

For a window manager during setup, I did not select Gnome or KDE.  I used XFCE.  I don’t see the need for a hugely fat window manager on a media center.  XFCE does the job quite nicely.

For RPM repositories, I elected to use RPMfusion.  The choice was more stylistic than anything, but it’s very much recommended you use either RPMfusion or ATrpms, not both.

After basic setup, I had to head over to linuxtv.org, and follow the setup instructions for my tuners.  Note that Fedora already has the drivers for them all, I just needed to drop the firmware into /lib/firmware .

For the MythTV theme, I used Mythbuntu, primarily because it’s clean and easy to navigate, looks acceptable, and not too heavyweight.

Bzzt!  One tuner down!

It took me a while to notice, but it turns out that one of my tuners (both are Hauppauge HVR-2200’s based off the saa7164 driver) won’t load firmware – it says the image is corrupt and therefore doesn’t work.  Unfortunately, I don’t know if it ever worked under Windows.  It shares an IRQ with the graphics card, which may have something to do with it.

Anyway, after much stuffing about, I’ve called it a loss and pulled the card.  I’ve since put in a Leadtek Winfast DTV Dongle Gold instead to supply a third tuner.  Again, firmware from linuxtv.org and the drivers were already included in Fedora.  It “just worked”.

I’ll put Hauppauge tuner into one of the main PCs when I next have one out and see if it works at all.  At any rate, it doesn’t matter much because MythTV can take two channels out of the one multiplexed stream, and so I can potentially record up to six things at once, on up to three transponders (so, I could do ABC1/ABC2 + 9/GO + 10/ONE or similar).

Programme Guides

I was previously a customer of IceTV because they offer an excellent EPG service for WIndows Media Center users.  I definitely recommend them for WMC users.  But now I’m not a WMC user, so I swapped over to Shepherd instead.  Shepherd requires a small amount of setting up, but now it’s configured the data I’m getting looks pretty clean and useable.

I’ll see how it goes compared to IceTV.

Autostart Configuration

In order to get MythTV to autostart on system boot, you configure /etc/gdm/custom.conf , and in the [daemon] block, add the following;

TimedLoginEnable=true
TimedLogin=mythtv
TimedLoginDelay=0

That will make your mythtv user auto-login.  Then, in XFCE, go to Settings -> Session and Startup, and add mythwelcome to the startup.  On boot, your MythTV frontend should start up.

Auto-Kill Frontend with IR Remote

First, put irexec -d in your XFCE startup.  That will cause your remote to be operatable outside of Myth.  Well, first set up your remote following the LIRC instructions, but anyway.  Make a .lircrc like this;

begin
    remote = mceusb
    prog = irexec
    button = Power
    config = /usr/local/bin/killmythfrontend.sh
    repeat = 0
    delay = 0
end

And then make that killmythfrontend.sh kill the frontend with “killall mythfrontend”.  Now, pressing the Power button your remote will kill your front end instantly.  Mythwelcome should then start it back up again.

Graphic Artifacts in VDPAU

I’ve noticed when using VDPAU (nVidia hardware accelerated video decoding) that I seem to be seeing visual artifacts – specifically glitches of white and red on the surfaces of things that are moving.  When I disable VDPAU I don’t see that, but I want to use VDPAU.  So, I’ve gone and grabbed a new GT520 video card, and I’ll see how that goes.  It might also be a driver issue, so I’ll have to find out.

Digital Audio – A Lesson in Pain

Ouch.  I had a devil of a time getting digital audio working.  I wanted to get digital audio going out my coax port to my amp (which supports DTS).  Sounds pretty easy, right?  Not on your life.

First I got digital audio working!  Yay!  But then I noticed that I couldn’t control the volume.  And then I noticed that it was only stereo.  DOH!  Then I got volume control working.  And then I got surround working.  But then I noticed that I couldn’t play anything that had AC3 audio.  DOH!  Then I got AC3 working, but then I noticed that the volume control was broken again.

Finally, I gave up and ran the analog output of my HTPC into my amp and just ran off that.  It works great.  Sigh.

What I want to see about doing is whether my TV can do digital passthrough out, because then I could just run the amp from the TV and it would “auto-select” what channel to run off.  I might see about that when I feel up to it.

I imagine half the problem is me just having little understanding of how digital audio works.  I might have to settle with having no volume control off the remote (and only on the amp), and then configure an IR blaster to control the amp volume and mute that way.  That’s probably the “right” way.

Results As Of Now

As of now, I’ve got my MythTV box set up so that it can record, play recorded TV, the remote control works, and DPMS monitor suspend is operational so it’ll shut down the screen.  My video library has been moved across to it (although I haven’t fully done the renaming and metadata collection for everything).  EPG is set up and working.  I still need to get automated sleep and wakeup working.

Still remaining is making another foray into digital audio.  I’m not looking forward to that.