diff --git a/sites/mytelly.co.uk/__data__/content.html b/sites/mytelly.co.uk/__data__/content.html index 1732193d..5dc21205 100644 --- a/sites/mytelly.co.uk/__data__/content.html +++ b/sites/mytelly.co.uk/__data__/content.html @@ -1,1076 +1,838 @@ - - - BBC One London - TV Listings Guide + + +BBC One London - TV Listings Guide - - - - - + + + + + - - - - - - + + + + + + + + - - - - + + + + - + - - - - - + + + + - - + - - - + gtag('config', 'G-5YFXQELX9H'); + + +
- - - - -
- -
-
- -
-
- BBC One London -
-

BBC One London

-
- Live -
- Find out what's on BBC One London tonight at the - UK TV Listings Guide -
-
-
- -
-
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
Time
-
-
TV Show
-
-
12:15 am
-
-
- - The Rap Game UK -
-
-
01:20 am
-
-
- - The Graham Norton Show -
-
-
02:13 am
-
-
- - Weather for the Week Ahead -
-
-
02:15 am
-
-
- - BBC News -
-
-
06:00 am
-
-
- - Breakfast -
-
-
07:30 am
-
-
- - Match of the Day -
-
-
09:00 am
-
-
- - Sunday with Laura Kuenssberg -
-
-
10:00 am
-
-
- - Politics London -
-
-
10:30 am
-
-
- - Animal Park -
-
-
11:00 am
-
-
- - Amanda & Alan's Italian Job -
-
-
11:30 am
-
-
- - Amanda & Alan's Italian Job -
-
-
12:00 pm
-
-
- - Bargain Hunt -
-
-
1:00 pm
-
-
- - BBC News -
-
-
1:10 pm
-
-
- - Weather for the Week Ahead -
-
-
1:15 pm
-
-
- - Songs of Praise -
-
-
2:00 pm
-
-
- - Planet Earth III -
-
-
3:00 pm
-
-
- - Live Snooker: UK Championship -
-
-
4:55 pm
-
-
- - BBC News -
-
-
5:10 pm
-
-
- - BBC London News; Weather -
-
-
5:15 pm
-
-
- - Weather -
-
-
5:20 pm
-
-
- - Countryfile -
-
-
6:20 pm
-
-
- - Planet Earth III -
-
-
7:20 pm
-
-
- - Strictly Come Dancing -
-
-
8:00 pm
-
-
- - Survivor -
-
-
9:00 pm
-
-
- - Boat Story -
-
-
10:00 pm
-
-
- - BBC News; Weather -
-
-
10:25 pm
-
-
- - BBC London News; Weather -
-
-
10:30 pm
-
-
- - Match of the Day 2 -
-
-
11:30 pm
-
-
- - The Women's Football Show -
-
- - | - More channels at My Telly..
-

-
-
-
- -
- -
- -
-
-
-
- -
- + - - + + - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/sites/mytelly.co.uk/__data__/programme.html b/sites/mytelly.co.uk/__data__/programme.html new file mode 100644 index 00000000..1bf7baff --- /dev/null +++ b/sites/mytelly.co.uk/__data__/programme.html @@ -0,0 +1,814 @@ + + + + + + Captain Phillips - TV Listings | My Telly + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+
+
+
+
+
+
+
+
+
+
+
+

+ Captain Phillips +

+
+
+
+
+ + + + +
12:00 am - 02:05 am, + Today + on BBC + One London (1) +
+
+
+ + + + +
+
+

+ + + + + +

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ Average User Rating: 4.50 (2 votes)
+ My Rating: Sign in or + Register to view last vote +

+
+
+ + + Add to Favorites + +
+
+
+
+
+ +
+
+
+
+
+

About this Broadcast

- +
+
+ +
+
+

An American cargo ship sets a dangerous course around the coast of Somalia, while inland, + four men are pressed into service as pirates by the local warlords. The captain is taken + hostage when the raiding party hijacks the vessel, resulting in a tense five-day crisis. + Fact-based thriller, starring Tom Hanks and Barkhad Abdi

+
+
+ 2013 + HD + subtitles + 16x9 +
+
+ Factual + Movie/Drama + Thriller +
+
+

+

+

+ Cast & Crew +

- +

+
+
+
+ + Tom Hanks + + (Actor) + .. Capt Richard Phillips +
+
+
+
+ + Barkhad Abdi + + (Actor) + .. Muse +
+
+
+
+ + Catherine Keener + + (Actor) + .. Andrea Phillips +
+
+
+
+ + Barkhad Abdirahman + + (Actor) + .. Bilal +
+
+
+
+ + Faysal Ahmed + + (Actor) + .. Najee +
+
+
+
+ + Mahat M Ali + + (Actor) + .. Elmi +
+
+
+
+ + Michael Chernus + + (Actor) + .. Shane Murphy +
+
+
+
+ + David Warshofsky + + (Actor) + .. Mike Perry +
+
+
+
+ + Paul Greengrass + + (Director) +
+
+
+ + + +
+

+

+

More Information

- +

+ +
No Logo
+
+ +
No Logo
+
+
+
+
+
+
+ + + + + + + +
+
+
+
+
+ +
+

+ Did You Know.. +

-

+
+
+
+
+ Tom Hanks + (Actor) + .. Capt Richard Phillips +
+
+
+ + Born: + + July 09, 1956 + in + California +
+
+ + Best Known For: + + Winning Oscars for Forrest Gump and Philadelphia +
+
+ + Early-life: + + Born in California in 1956, Tom Hanks grew up in what he calls a "fractured" + family. His parents were pioneers in the development of marriage dissolution law in that + state, and Tom moved around a lot, living with a succession of step-families. Eager for + a career in acting, he auditioned for a community theater play, was invited by the + director of that play to go to Cleveland, and there his acting career started. +
+
+ + Career: + + After several TV roles, Hanks made the transition to movies and became one of the + biggest stars of the past three decades. His breakthrough came in 1984's Splash, + and was followed by hit after hit, including Bachelor Party and Dragnet. When he took + the lead in Oscar-nominated Big in 1988, he transformed a simple comedy into something + much more powerful, before truly proving his acting credentials with an Oscar-winning + turn as an Aids-afflicted laweyer in Philadeplhia. He gave terrific turns in romantic + comedies like Sleepless in Seattle, then another Oscar followed for Forrest Gump. Many + expected him to garner a third for Saving Private Ryan. He has been a successful + director of several projects and helped bring iconic TV projects like Band of Brothers + to the screen. Recent hits include The Da Vinci Code and its follow-up, and the Toy + Story films. +
+
+ + Quote: + + "My wife keeps on telling me my worst fault is that I keep things to myself and + appear relaxed. But I am really in a room in my own head and not hearing a thing anyone + is saying." +
+
+
+
+
+ Barkhad Abdi + (Actor) + .. Muse +
+
+
+
+ Catherine Keener + (Actor) + .. Andrea Phillips +
+
+
+ + Born: + + March 23, 1959 + in + Miami, Florida +
+
+ + Best Known For: + + Playing Maxine Lund in Being John Malkovich. +
+
+ + Early-life: + + Catherine Ann Keener was born in Miami, Florida on March 23, 1959. She has four + siblings, including actress Elizabeth Keener. Catherine studied at Wheaton College in + Massachusetts, where she graduated with a degree in English and history in 1983. While + studying here, she joined a theatre course and appeared in her first theatrical + production during her junior year. +
+
+ + Career: + + Keener's first film appearance was a one-worded role as a waitress in About Last + Night… (1986). She then went on to appear in Survival Quest (1988). After struggling in + the industry for a number of years, she landed a part in Johnny Suede (1991) and began + getting roles in independent films. Her mainstream breakthrough was in Being John + Malkovich (1999). Since then, she has appeared in many more films, including Capote + (2005), Into The Wild (2007), Captain Phillips (2013) and Get Out (2017). +
+
+ + Quote: + + "Luck favours the prepared." +
+
+ + Trivia: + + She was Oscar-nominated for Best Supporting Actress in 2000 for her performance in Being + John Malkovich and in 2006 for her performance in Capote. +
+
+
+
+
+ Barkhad Abdirahman + (Actor) + .. Bilal +
+
+
+
+ Faysal Ahmed + (Actor) + .. Najee +
+
+
+
+ Mahat M Ali + (Actor) + .. Elmi +
+
+
+
+ Michael Chernus + (Actor) + .. Shane Murphy +
+
+
+
+ David Warshofsky + (Actor) + .. Mike Perry +
+
+
+
+ Paul Greengrass + (Director) +
+
+
+
+
+
+
+
+ +
+
+
+
+
+

Before / After

- +

+ + + + + +
+ + + + + +
+ + + +
+
+ + + + + +
+ + + +
+
+
+
+
+
+ + +
+ +
+ +
+
+
+
+ + + +
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/sites/mytelly.co.uk/__data__/programme2.html b/sites/mytelly.co.uk/__data__/programme2.html new file mode 100644 index 00000000..e1c637d2 --- /dev/null +++ b/sites/mytelly.co.uk/__data__/programme2.html @@ -0,0 +1,796 @@ + + + + + + The Rap Game UK - TV Listings | My Telly + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+
+
+
+
+
+
+
+
+
+
+
+

+ The Rap Game UK +

+
+
+
+
+ + + + +
11:35 pm - 12:40 am, + Today + on BBC + One Channel Islands (1) +
+
+
+ + + + +
+
+

+ + + + + +

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ Average User Rating: 1.00 (1 + votes)
+ My Rating: Sign in or + Register + to view last vote +

+
+
+ + + Add to Favorites + +
+
+
+
+
+ +
+
+
+
+
+

About this Broadcast

- +
+
+ +
+
+ Past and Pressure
+ Season 6, Episode 5 +
+
+

The artists are tasked with writing a song about their heritage. + For some, the pressure of the competition proves too much for + them to match. In their final challenge, they are put face to + face with industry experts who grill them about their plans + after the competition. Some impress, while others leave the + mentors confused

+
+
+ HD + subtitles + repeat + 16x9 +
+
+ Challenge/Reality Show + Show/Game Show +
+
+

+

+

+ Cast & Crew +

- +

+
+
+
+ + Konan + + (Presenter) +
+
+
+
+ + Krept + + (Presenter) +
+
+
+
+ + DJ Target + + (Presenter) +
+
+
+
+ + Hetal Dhanak + + (Series producer) +
+
+
+
+ + Tom O'Brien + + (Executive producer) +
+
+
+
+ + Paul Broadbent + + (Executive producer) +
+
+
+
+ + Trent Williams-Jones + + (Executive producer) +
+
+
+ + + +
+

+

+

More Information

- +

+ +
No Logo
+
+
+
+
+
+
+ + + + + +
+
+
+
+
+ +
+

Upcoming Broadcasts

-

+
+

The Rap Game UK is scheduled to broadcast again at the + following times

+
+
+

+ +
+
Saturday, December 07
+
+ + + + + + + + + + + + + + + +
+
Time
+
+
Episode
+
+
Channel
+
+
11:35 pm
+
+
+ The Rap Game UK +
+
+ Season: 6, + Episode: 5 +
+
+ The artists are tasked with writing a song about + their heritage. For some, the pressure of the + competition proves too much for them to match. In + their final challenge, they are put face to face + with industry experts who grill them about their + plans after the competition. Some impress, while + others leave the mentors confused +
+
+
+ + BBC One Channel Islands HD + (101) + +
+
+ (BT TV) +
+
+
+
+
+ + +
+
+
+
+
+ +
+

+ Did You Know.. +

-

+
+
+
+
+ Konan + (Presenter) +
+
+
+
+ Krept + (Presenter) +
+
+
+
+ DJ Target + (Presenter) +
+
+
+
+ Hetal Dhanak + (Series producer) +
+
+
+
+ Tom O'Brien + (Executive producer) +
+
+
+
+ Paul Broadbent + (Executive producer) +
+
+
+
+ Trent Williams-Jones + (Executive producer) +
+
+
+
+
+
+
+
+ +
+
+
+
+
+

Before / After

- +

+ + + + + +
+ + + + + +
+ + + +
+
+ + + + + +
+ + + +
+
+
+
+
+
+ + +
+ +
+ +
+
+
+
+ + + +
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/sites/mytelly.co.uk/mytelly.co.uk.channels.xml b/sites/mytelly.co.uk/mytelly.co.uk.channels.xml index f3f2440a..f077129c 100644 --- a/sites/mytelly.co.uk/mytelly.co.uk.channels.xml +++ b/sites/mytelly.co.uk/mytelly.co.uk.channels.xml @@ -1,6 +1,89 @@ - 4Music + MTV XMAS + Phoenix CNE Channel + U&Dave + Bloomberg + U&Gold + U&Alibi + Nicktoons + U&Gold +1 + U&Yesterday + U&Eden + U&Eden +1 + Legend Xtra + U&Yesterday +1 + U&Dave ja vu + GREAT! christmas + U&Alibi +1 + Sky Cinema 80s + Sky Cinema Favourites + Sky Cinema Epics + GREAT! christmas +1 + Legend Xtra +1 + U&W + Sky Cinema Favourites HD + U&W +1 + True Crime + Sky Cinema Epics HD + True Crime +1 + True Crime Xtra + Nick Jr. Too + Sky Cinema 80s HD + U&Eden HD + LFC TV + Premier Sports 1 HD + U&W HD + U&Dave HD + BBC One Yorkshire & Lincolnshire + U&Alibi HD + U&Drama + Brit Asia + RTE One HD + Leeds TV + Bristol TV + Tyne & Wear TV + Cardiff TV + NDTV World + Arirang + Birmingham TV + Liverpool TV + Blaze +1 (Freeview) + New Vision TV + Sky Sports + + Sky Sports + HD + U&Gold HD + TNT Sports Box Office + ARY Digital + HUM Europe + Premier Sports 2 HD + U&Drama +1 + TNT Sports Box Office 2 + Talk + U&Yesterday HD + That's Christmas + U&Drama HD + That's Melody MCR + That's Melody + Cula4 + That's TV MCR + That's Fabulous MCR + That's Fabulous + That's TV 2 + North Wales TV + Teesside TV + ITV1 Channel Television HD + ITV1 Border Scotland HD + Sky Sports Tennis + Sky Sports Tennis HD + Rewind TV + Music India + NOW XMAS + That's Christmas + That's TV 2 MCR + That's Dance (Freeview) + That's Dance (MCR and Sky) + Cartoonito HD 4seven 4seven HD 5Action @@ -10,16 +93,11 @@ 5STAR +1 5USA 5USA +1 - Alibi - Alibi HD - Alibi +1 Al Jazeera English Al Jazeera English HD Animal Planet Animal Planet HD Animal Planet +1 - Arirang HD - Ayozat TV B4U Movies B4U Music Baby TV @@ -38,7 +116,6 @@ BBC One East HD BBC One East Midlands BBC One East Midlands HD - BBC One HD BBC One London BBC One London HD BBC One North East & Cumbria @@ -96,8 +173,6 @@ CBBC HD CBeebies CBeebies HD - CBS Reality - CBS Reality +1 Challenge Channel 4 Channel 4 HD @@ -126,9 +201,6 @@ Crime + Investigation HD Crime + Investigation +1 Cula4 HD - Dave - Dave HD - Dave ja vu Discovery Channel Discovery Channel HD Discovery Channel +1 @@ -140,19 +212,10 @@ Discovery Turbo +1 DMAX DMAX +1 - Drama - Drama HD - Drama +1 - E! Entertainment E4 E4 Extra E4 HD E4 +1 - Earthx TV - Eden - Eden HD - Eden +1 - E! Entertainment HD Euronews Eurosport 1 Eurosport 1 HD @@ -163,28 +226,20 @@ Film4 +1 Food Network Food Network +1 - Foodxp France 24 English France 24 English HD GB News GB News HD GEO News GEO TV - GOLD - GOLD HD - GOLD +1 GREAT! action GREAT! action +1 GREAT! movies GREAT! movies +1 - GREAT! movies christmas - GREAT! movies christmas +1 GREAT! tv GREAT! tv +1 HGTV HGTV +1 - HorrorXtra - HorrorXtra +1 Ideal World Investigation Discovery Investigation Discovery +1 @@ -235,20 +290,16 @@ ITVBe ITVBe HD ITVBe +1 - Kerrang! TV - Kiss TV Legend LFC TV HD London Live - Magic More4 More4 HD More4 +1 Christmas 24 Christmas 24+ - MTV MTV 80s - MTV 90s + MTV MTV HD MTV Hits MTV Live HD @@ -260,8 +311,6 @@ National Geographic Channel +1 National Geographic WILD National Geographic WILD HD - NDTV 24x7 - New Vision NHK World NHK World HD Nickelodeon @@ -270,8 +319,6 @@ Nick Jr. Nick Jr. HD Nick Jr. +1 - Nick Jr. Peppa - NickLoudHouse Notts TV NOW 70s NOW 80s @@ -288,8 +335,8 @@ Quest +1 Quest Red Quest Red +1 - QVC QVC2 + QVC QVC Beauty QVC Extra QVC HD @@ -297,7 +344,6 @@ QVC Style HD Racing TV Racing TV HD - RealityXtra Really RTE2 RTE2 HD @@ -322,17 +368,10 @@ Sky Cinema Christmas HD Sky Cinema Family Sky Cinema Family HD - Sky Cinema Greats - Sky Cinema Greats HD - Sky Cinema Bourne - Sky Cinema Hits HD Sky Cinema Premiere Sky Cinema Premiere HD - Sky Cinema Premiere +1 Sky Cinema Sci-fi/Horror Sky Cinema Sci-fi/Horror HD - Sky Cinema Heroines - Sky Cinema Heroines HD Sky Cinema Thriller Sky Cinema Thriller HD Sky Comedy @@ -342,9 +381,9 @@ Sky Crime +1 Sky Documentaries Sky Documentaries HD - Sky History Sky History 2 Sky History 2 HD + Sky History Sky History HD Sky History +1 Sky Kids @@ -363,9 +402,6 @@ Sky Showcase Sky Showcase HD Sky Showcase +1 - Sky Sports Arena - Sky Sports Arena HD - Sky Sports Box Office Sky Sports Cricket Sky Sports Cricket HD Sky Sports F1 @@ -396,45 +432,21 @@ Sony MAX Sony MAX HD Sony SAB - Sportystuff TV HD STV Central STV Central +1 STV HD STV North STV North +1 - TalkBirmingham - TalkBristol - TalkCardiff Talking Pictures TV - TalkLeeds - TalkLiverpool - TalkNorthWales - TalkTeesside - Talk TV - Talk TV HD - TalkTyne&Wear TG4 TG4 HD - That's 60s - That's 60s (MCR Freeview) - That's Christmas - That's 70s (MCR Freeview) - That's 80s - That's 80s (Freeview) - That's 90s - That's 90s (MCR Freeview) That's TV - That's TV (Freeview) - The Box Tiny Pop - Tiny Pop (Freeview) Tiny Pop +1 TLC TLC HD TLC +1 TNT Sports 1 - TNT Sports 10 - TNT Sports 10 HD TNT Sports 1 HD TNT Sports 2 TNT Sports 2 HD @@ -452,14 +464,13 @@ TNT Sports 8 HD TNT Sports 9 TNT Sports 9 HD - TNT Sports Box Office HD 2 + TNT Sports 10 + TNT Sports 10 HD Sky Sports Box Office HD - TNT Sports Box Office HD TNT Sports Ultimate TNT Sports Ultimate Together TV Together TV +1 - Trace Vault Travelxp TRT World TRT World HD @@ -472,16 +483,7 @@ UTV UTV HD UTV +1 - Viaplay Sports 1 HD - Viaplay Sports 2 HD - Viaplay Xtra HD - W - W HD WildEarth - W +1 - Yesterday - Yesterday HD - Yesterday +1 Zee Cinema HD Zee Cinema Zee TV HD diff --git a/sites/mytelly.co.uk/mytelly.co.uk.config.js b/sites/mytelly.co.uk/mytelly.co.uk.config.js index 7272a0a6..f08be0bd 100644 --- a/sites/mytelly.co.uk/mytelly.co.uk.config.js +++ b/sites/mytelly.co.uk/mytelly.co.uk.config.js @@ -3,131 +3,255 @@ const dayjs = require('dayjs') const utc = require('dayjs/plugin/utc') const timezone = require('dayjs/plugin/timezone') const customParseFormat = require('dayjs/plugin/customParseFormat') +const debug = require('debug')('site:mytelly.co.uk') dayjs.extend(utc) dayjs.extend(timezone) dayjs.extend(customParseFormat) +const detailedGuide = true +const tz = 'Europe/London' +const nworker = 25 + module.exports = { site: 'mytelly.co.uk', days: 2, - url: function ({ date, channel }) { + url({ date, channel }) { return `https://www.mytelly.co.uk/tv-guide/listings/channel/${ channel.site_id }.html?dt=${date.format('YYYY-MM-DD')}` }, - parser: function ({ content, date, channel }) { + async parser({ content, date, channel }) { const programs = [] - const items = parseItems(content) - items.forEach(item => { - const prev = programs[programs.length - 1] - const $item = cheerio.load(item) - let start = parseStart($item, date, channel) - if (prev) { - if (start.isBefore(prev.start)) { - start = start.add(1, 'd') - date = date.add(1, 'd') - } - prev.stop = start - } - const stop = start.add(30, 'm') - programs.push({ - title: parseTitle($item), - start, - stop + + if (content) { + const queues = [] + const $ = cheerio.load(content) + + $('table.table > tbody > tr').toArray() + .forEach(el => { + const td = $(el).find('td:eq(1)') + const title = td.find('h5 a') + if (detailedGuide) { + queues.push(title.attr('href')) + } else { + const subtitle = td.find('h6') + const time = $(el).find('td:eq(0)') + let start = parseTime(date, time.text().trim()) + const prev = programs[programs.length - 1] + if (prev) { + if (start.isBefore(prev.start)) { + start = start.add(1, 'd') + date = date.add(1, 'd') + } + prev.stop = start + } + const stop = start.add(30, 'm') + programs.push({ + title: parseText(title), + subTitle: parseText(subtitle), + start, + stop + }) + } + }) + + if (queues.length) { + await doFetch(queues, (url, res) => { + const $ = cheerio.load(res) + const time = $('center > h5 > b').text() + const title = parseText($('.inner-heading.sub h2')) + const subTitle = parseText($('.tab-pane > h5 > strong')) + const description = parseText($('.tab-pane > .tvbody > p')) + const image = $('.program-media-image img').attr('src') + const category = $('.schedule-attributes-genres span').toArray() + .map(el => $(el).text()) + const casts = $('.single-cast-head:not([id])').toArray() + .map(el => { + const cast = { name: parseText($(el).find('a')) } + const [, role] = $(el).text().match(/\((.*)\)/) || [null, null] + if (role) { + cast.role = role + } + return cast + }) + const [start, stop] = parseStartStop(date, time) + let season, episode + if (subTitle) { + const [, ses, epi] = subTitle.match(/Season (\d+), Episode (\d+)/) || [null, null] + if (ses) { + season = parseInt(ses) + } + if (epi) { + episode = parseInt(epi) + } + } + programs.push({ + title, + subTitle, + description, + image, + category, + season, + episode, + actor: casts.filter(c => c.role === 'Actor').map(c => c.name), + director: casts.filter(c => c.role === 'Director').map(c => c.name), + presenter: casts.filter(c => c.role === 'Presenter').map(c => c.name), + start, + stop + }) }) - }) + } + } return programs }, async channels() { + const channels = {} const axios = require('axios') - const _ = require('lodash') - - const providers = [ - '-3000053', - '-4000118', - '-11000199', - '-1000007', - '-2000007', - '-12000220', - '-5000136', - '-10000178' - ] - const regions = [ - 'Cambridgeshire', - 'Channel Islands', - 'Cumbria', - 'East', - 'East Midlands', - 'Yorkshire & Lincolnshire', - 'London', - 'North East', - 'North West', - 'Northern Ireland', - 'Oxfordshire', - 'Scotland (Borders)', - 'Scotland (Central)', - 'Scotland (North)', - 'South', - 'South East', - 'South West', - 'Wales', - 'West', - 'West Midlands', - 'Yorkshire' - ] - - const channels = [] - for (let provider of providers) { - for (let region of regions) { - const data = await axios - .post(`https://www.mytelly.co.uk/tv-guide/schedule`, null, { - params: { - provider, - region, - TVperiod: 'Night', - date: dayjs().format('YYYY-MM-DD'), - st: 0, - u_time: 1955, - is_mobile: 1 + const queues = [{ t: 'p', m: 'post', u: 'https://www.mytelly.co.uk/getform' }] + await doFetch(queues, (queue, res) => { + // process form -> provider + if (queue.t === 'p') { + const $ = cheerio.load(res) + $('#guide_provider option').toArray() + .forEach(el => { + const opt = $(el) + const provider = opt.attr('value') + queues.push({ t: 'r', m: 'post', u: 'https://www.mytelly.co.uk/getregions', params: { provider } }) + }) + } + // process provider -> region + if (queue.t === 'r') { + const now = dayjs() + for (const r of Object.values(res)) { + const params = { + provider: queue.params.provider, + region: r.title, + TVperiod: 'Night', + date: now.format('YYYY-MM-DD'), + st: 0, + u_time: now.format('HHmm'), + is_mobile: 1 + } + queues.push({ t: 's', m: 'post', u: 'https://www.mytelly.co.uk/tv-guide/schedule', params }) + } + } + // process schedule -> channels + if (queue.t === 's') { + const $ = cheerio.load(res) + $('.channelname') + .each((i, el) => { + const name = $(el).find('center > a:eq(1)').text() + const url = $(el).find('center > a:eq(1)').attr('href') + const [, number, slug] = url.match(/\/(\d+)\/(.*)\.html$/) + const site_id = `${number}/${slug}` + if (channels[site_id] === undefined) { + channels[site_id] = { + lang: 'en', + site_id, + name + } } }) - .then(r => r.data) - .catch(console.log) - - const $ = cheerio.load(data) - $('.channelname').each((i, el) => { - const name = $(el).find('center > a:eq(1)').text() - const url = $(el).find('center > a:eq(1)').attr('href') - const [, number, slug] = url.match(/\/(\d+)\/(.*)\.html$/) - - channels.push({ - lang: 'en', - name, - site_id: `${number}/${slug}` - }) - }) } - } + }) - return _.uniqBy(channels, 'site_id') + return Object.values(channels) } } -function parseStart($item, date, channel) { - const timeString = $item('td:eq(0)').text().trim() - const dateString = `${date.format('YYYY-MM-DD')} ${timeString}` +function parseStartStop(date, time) { + const [s, e] = time.split(' - ') + const start = parseTime(date, s) + let stop = parseTime(date, e) + if (stop.isBefore(start)) { + stop = stop.add(1, 'd') + } - return dayjs.tz(dateString, 'YYYY-MM-DD H:mm a', 'Europe/London') + return [start, stop] } -function parseTitle($item) { - return $item('td:eq(1)').text().trim() +function parseTime(date, time) { + return dayjs.tz(`${date.format('YYYY-MM-DD')} ${time}`, 'YYYY-MM-DD H:mm a', tz) } -function parseItems(content) { - const $ = cheerio.load(content) +function parseText($item) { + let text = $item.text() + .replace(/\t/g, '') + .replace(/\n/g, ' ') + .trim() + while (true) { + if (text.match(/ /)) { + text = text.replace(/ /g, ' ') + continue + } + break + } - return $('table.table > tbody > tr').toArray() + return text } + +async function doFetch(queues, cb) { + const axios = require('axios') + + let n = Math.min(nworker, queues.length) + const workers = [] + const adjustWorker = () => { + if (queues.length > workers.length && workers.length < nworker) { + let nw = Math.min(nworker, queues.length) + if (n < nw) { + n = nw + createWorker() + } + } + } + const createWorker = () => { + while (workers.length < n) { + startWorker() + } + } + const startWorker = () => { + const worker = () => { + if (queues.length) { + const queue = queues.shift() + const done = res => { + if (res) { + cb(queue, res) + adjustWorker() + } + worker() + } + const url = typeof queue === 'string' ? queue : queue.u + const params = typeof queue === 'object' && queue.params ? queue.params : {} + const method = typeof queue === 'object' && queue.m ? queue.m : 'get' + debug(`fetch %s with %s`, url, JSON.stringify(params)) + if (method === 'post') { + axios + .post(url, params) + .then(response => done(response.data)) + .catch(console.error) + } else { + axios + .get(url, params) + .then(response => done(response.data)) + .catch(console.error) + } + } else { + workers.splice(workers.indexOf(worker), 1) + } + } + workers.push(worker) + worker() + } + createWorker() + await new Promise(resolve => { + const interval = setInterval(() => { + if (workers.length === 0) { + clearInterval(interval) + resolve() + } + }, 500) + }) +} + diff --git a/sites/mytelly.co.uk/mytelly.co.uk.test.js b/sites/mytelly.co.uk/mytelly.co.uk.test.js index ac368528..20f6ced0 100644 --- a/sites/mytelly.co.uk/mytelly.co.uk.test.js +++ b/sites/mytelly.co.uk/mytelly.co.uk.test.js @@ -1,47 +1,81 @@ const { parser, url } = require('./mytelly.co.uk.config.js') +const axios = require('axios') const fs = require('fs') const path = require('path') const dayjs = require('dayjs') const utc = require('dayjs/plugin/utc') const customParseFormat = require('dayjs/plugin/customParseFormat') + dayjs.extend(customParseFormat) dayjs.extend(utc) -const date = dayjs.utc('2023-11-26', 'YYYY-MM-DD').startOf('d') +jest.mock('axios') + +const date = dayjs.utc('2024-12-07', 'YYYY-MM-DD').startOf('d') const channel = { site_id: '713/bbc-one-london', xmltv_id: 'BBCOneLondon.uk' } +axios.get.mockImplementation((url, opts) => { + if ( + url === 'https://www.mytelly.co.uk/tv-guide/listings/programme?cid=713&pid=1906433&tm=2024-12-07+00%3A00%3A00' + ) { + return Promise.resolve({ + data: fs.readFileSync(path.join(__dirname, '__data__', 'programme.html')) + }) + } + if ( + url === 'https://www.mytelly.co.uk/tv-guide/listings/programme?cid=713&pid=5656624&tm=2024-12-07+23%3A35%3A00' + ) { + return Promise.resolve({ + data: fs.readFileSync(path.join(__dirname, '__data__', 'programme2.html')) + }) + } + + return Promise.resolve({ data: '' }) +}) + it('can generate valid url', () => { expect(url({ channel, date })).toBe( - 'https://www.mytelly.co.uk/tv-guide/listings/channel/713/bbc-one-london.html?dt=2023-11-26' + 'https://www.mytelly.co.uk/tv-guide/listings/channel/713/bbc-one-london.html?dt=2024-12-07' ) }) -it('can parse response', () => { - const content = fs.readFileSync(path.resolve(__dirname, '__data__/content.html')) - const results = parser({ content, channel, date }).map(p => { +it('can parse response', async () => { + const content = fs.readFileSync(path.join(__dirname, '__data__', 'content.html')) + const results = (await parser({ content, channel, date })).map(p => { p.start = p.start.toJSON() p.stop = p.stop.toJSON() return p }) + expect(results.length).toBe(2) expect(results[0]).toMatchObject({ - start: '2023-11-26T00:15:00.000Z', - stop: '2023-11-26T01:20:00.000Z', - title: 'The Rap Game UK' + start: '2024-12-07T00:00:00.000Z', + stop: '2024-12-07T02:05:00.000Z', + title: 'Captain Phillips', + description: + 'An American cargo ship sets a dangerous course around the coast of Somalia, while inland, four men are pressed into service as pirates by the local warlords. The captain is taken hostage when the raiding party hijacks the vessel, resulting in a tense five-day crisis. Fact-based thriller, starring Tom Hanks and Barkhad Abdi', + image: 'https://d16ia5iwuvax6y.cloudfront.net/uk-prog-images/c44ce7b0d3ae602c0c93ece5af140815.jpg?k=VeeNdUjml3bSHdlZ0OXbGLy%2BmsLdYPwTV6iAxGkzq4dsylOCGGE7OWlqwSWt0cd0Qtrin4DkEMC0Zzdp8ZeNk2vNIQzjMF0DG0h3IeTR5NM%3D', + category: ['Factual', 'Movie/Drama', 'Thriller'] }) - - expect(results[28]).toMatchObject({ - start: '2023-11-26T23:30:00.000Z', - stop: '2023-11-27T00:00:00.000Z', - title: "The Women's Football Show" + expect(results[1]).toMatchObject({ + start: '2024-12-07T23:35:00.000Z', + stop: '2024-12-08T00:40:00.000Z', + title: 'The Rap Game UK', + subTitle: 'Past and Pressure Season 6, Episode 5', + description: + 'The artists are tasked with writing a song about their heritage. For some, the pressure of the competition proves too much for them to match. In their final challenge, they are put face to face with industry experts who grill them about their plans after the competition. Some impress, while others leave the mentors confused', + image: 'https://d16ia5iwuvax6y.cloudfront.net/uk-prog-images/2039278182b27cc279570b9ab9b89379.jpg?k=VeeNdUjml3bSHdlZ0OXbGLy%2BmsLdYPwTV6iAxGkzq4cDhR7jXTNFW3tgwQCdOPUobhXwlT81mIsqOe93HPusDG6tw1aoeYOgafojtynNWxc%3D', + category: ['Challenge/Reality Show', 'Show/Game Show'], + season: 6, + episode: 5 }) }) -it('can handle empty guide', () => { - const result = parser({ +it('can handle empty guide', async () => { + const result = await parser({ date, channel, content: ''