INTERACT FORUM

More => Old Versions => JRiver Media Center 27 for Windows => Topic started by: jmone on December 19, 2020, 06:04:08 pm

Title: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on December 19, 2020, 06:04:08 pm
Not a new suggestion, but last night I tried to share a home video with a friend and they stream everything these days.  I had to post my video on my Nextcloud Site, so they could download it, then copy to a USB Stick, then put it in the TV, then run up some app, then play it.  It worked but is ugly.

It does not need to be this way as almost all the components are already in MC to allow "clients" to stream media with the exception of the ability to create a "Public URL" to an item so it can be accessed by an browser anywhere in the word.... just like YouTube except it streams from your own MC Instance (and not a "Cloud").
- Media Server Profiles for transocoding already exists
- Media [url=http://wiki.jriver.com/index.php/Network_Access]Network Access (http://wiki.jriver.com/index.php/Network_Access)[/url] all sorted
- Web Services engine already exists

... All we would need is the ability for MC to be able to generate a URL that can be shared and then use that link to either download or play the file.


Here is an example of how my NextCloud instance does it (but it's video transcoding is non existent):
https://behome.dyndns.info/index.php/s/wrXHafPiscg5wt8


Thanks
Nathan

[Edit by JimH -- This was implemented and is in 27.0.66 or higher:  https://yabb.jriver.com/interact/index.php/topic,128554.msg892107.html#msg892107 ]
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: rpalmer68 on December 22, 2020, 01:54:21 am
I was hoping for something a little more exciting Nathan!

But yes I agree, it would be a handy feature at times.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on December 22, 2020, 03:29:45 am
Hey Richard - Long Time!

I think it is actually a big thing.  More and More people are going online only for media consumption.  I think MC should join in and become an server for such consumption over the web.  The family we visited used to be a MC user but now just stream everything, they simply don't have a local "player", so it makes sense that I should be able to just send them a URL that can play.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on December 22, 2020, 03:58:15 am
I should also say, that like many today we have plenty of upload bandwidth to be able to stream out video (well mine is only 50mbps but it is "good enough").
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on December 22, 2020, 04:10:45 am
It's a nice idea and certainly very useful, even for streaming our content ourselves when we're on vacation or outside the house. You can already do that with 3rd party software, not integrated into MC.

However, there are some additional considerations that make it a bit dangerous to share content:
- upload speed limit: most internet connections are asymmetric; the upload speed is much lower than the download. By streaming media, your upload can get saturated even with a single video stream, depending on quality. When that happens your DL also goes down the drain due to delayed ACKs.
- realtime transcoding sucks. Poor quality, and high CPU usage leading to fan noise and heat. Can't be done on a cheap HTPC, requires a good CPU or an expensive GPU. This can be avoided by doing direct streaming (no transcoding), but that only works for some video formats like MP4 (MKV won't work, for instance), and uses up more of that upload bandwidth.
- Sharing control: by giving a link to someone, how do you make sure that it won't get re-shared over and over again until you lose all control? Parents share the link to the daughter, who shares to a friend, who posts it on facebook; BAM, you now have 50 people accessing it, and there goes your internet connection until you block it.
- Related to the previous one, there are legal implications of sharing content. It's fine to share your home videos, but if you share a movie you legally bought and ripped, and then lose control and it gets widely shared or even torrented... tough luck, you're now liable.

I have remote access to my collection for when I'm on vacation, but I only use it for myself. When I need to share something to family/friends, I still do it the hard way, usually setting up a time-limited FTP/HTTP download, or uploading a home video to my (private) youtube channel.

One easy way to share content from your home is to set up an HTTP server. You can then have links to individual files, optional user/pass authentication, and direct streaming even works for MP4 and some other formats. Just... not integrated into MC.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on December 22, 2020, 04:14:25 am
I should also say, that like many today we have plenty of upload bandwidth to be able to stream out video (well mine is only 50mbps but it is "good enough").

Those 50Mbps upload are not the norm, by far. I have 500/25 and it's already an expensive service. Most people still have far less, or it's the "up to X" variety. Saturating your UL still kills your DL, unless you have a QoS router that prioritizes ACKs.

I agree though, if you have the upload bandwidth, why not make use of it...
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on January 25, 2021, 03:02:39 am
So we already can do all of this with Remotes from outside the LAN.... but you have to have access to the entire library with a Key/Username/Password to gain access.

All I'm suggesting is to use the existing services that already exist but allow the generation of a "public" URL that will allow access to that one item without requiring a log in.  Eg - this is what such a string looks like from Next Cloud : https://behome.dyndns.info/index.php/s/Fxc8GAZWtB2reYK (this is a real video of a red belly black snake at our front door.... we get a few!).  You can see, play, download this one Video but not the rest I have on the server. 

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on January 25, 2021, 06:42:55 am
I think this is a good idea.  Remind us from time to time.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on January 25, 2021, 02:04:25 pm
I do!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: RoderickGI on January 25, 2021, 05:09:51 pm
this is a real video of a red belly black snake at our front door.... we get a few!

Red Belly Blacks, not really a worry, unless you have a small dog. I see you do!

I have two Mini Foxies. They are hunters and think they are big dogs. The two most common ways they die are fighting larger dogs and snakebite.  :o
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on January 25, 2021, 05:43:55 pm
Yup - no fear at all.  Our little black dog tangled last year with a Red Belly and ended up with Multiple Strikes.  Got her to the Vet within 5 Min, multiple shots of antivenin, blood transfusion and a week in hospital. 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 04, 2021, 10:41:59 am
(deleted by Matt now that this is public)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 04, 2021, 10:59:04 am
Nice pond :)

But... that login allows us to access Panel, and via that we can see the entire collection.
Even just changing the File ID number on the URL works - File 24514624 also plays.

The idea is to share a single file at a time, with optional credentials, without giving access to Panel or to any other non-shared item.

You may want to change your access key now.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 04, 2021, 11:22:24 am
May I suggest an API for individual file sharing:
/MCWS/v1/Share?token=FileShareToken

- Token is a string encrypted with the server's AccessKey (or other server-defined key), and base64-encoded;
- optionally, the Token can embed an expiration timestamp or other restrictions;
- User can right-click on a File and select "Get Sharing URL", optionally providing an expiration date. MC generates the encrypted URL which can then be shared;
- MCWS API, on receiving the call, just needs to decrypt the token to check validity and extract the FileKey;
- Authentication is optional - but could be defined in Settings as a distinct "Sharing user/pass", or even a list of "user/pass" accounts.

You can make the token string as a JSON object, for instance, to allow for future features:
 Token: {File:24514625, Expiry:1612558645, Flags:0}
Encoded: mfhfiP5H0MMlO9id2YkWKzRFNRFe66lqTzkGvCHDf4gOp11O79k8ezCK
URL: http://73.164.109.246:52199/MCWS/v1/Share?token=mfhfiP5H0MMlO9id2YkWKzRFNRFe66lqTzkGvCHDf4gOp11O79k8ezCK

Flags is optional, up to you... it could for instance indicate if authentication is required or not for this URL.

Additionally, we could have a field called [Shared] (bool) which would become TRUE after sharing. Simply unchecking this field would revoke any existing URL pointing to it (MWCS just needs to check this field before serving the file)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 04, 2021, 12:27:59 pm
A simpler idea:
- [Shared] field, user can check/uncheck
- "Enable Sharing" in Settings, with a distinct User/Password
- Accessing MCWS/Panel will that account will only serve Shared files.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: kensn on February 04, 2021, 12:36:27 pm
Nice pond...

Worked for me.

Ken
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 04, 2021, 12:38:16 pm
A simpler idea:
- [Shared] field, user can check/uncheck
- "Enable Sharing" in Settings, with a distinct User/Password
- Accessing MCWS/Panel will that account will only serve Shared files.
I think we need to make it easy to share a file or files with a Share link that uses the standard icon. 

I agree that it needs to show only those files, so it would need to have no login / password necessary OR a different set.

It could also be a separate server.

We're on it though.  This is promising.  It solves a problem I've had for a long time.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 04, 2021, 12:43:33 pm
Adding what I just said to Matt in email:

Could it add files to a Shared Library and share anything in it with a different Access Key?  Or a [Named] Library so you could have different ones for different people?

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Awesome Donkey on February 04, 2021, 01:10:32 pm
Shame something like a two-factor authentication system couldn't be used here. Like if an outside user wants to access a shared library from outside the network, it could prompt the user in MC with a allow or deny choice. Which if there's some sort of manager, authentication allowing shared library access could be revoked or something like that.

Like I could create a guest access key, give it to somebody which when they use that access key within their copy of MC it'd connect and prompt me at my end informing me there's a connection attempt, that I can accept, temp allow it for X amount of time, deny, ignore, etc.

Just thinking out loud. :P
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 04, 2021, 01:23:54 pm
I think multiple libraries is problematic - updating the file tags in the main library would not update in the other libs, so this would require too much management overhead.

What about sharing playlists instead of files? We could have different playlists for different people, different credentials/key per playlist, etc. Seems more practical than distinct libraries. This could work for any generic playlist, or you could add a special Playlists\Shared section in the tree. Playlist Groups could also be used to share a group of playlists to the same person.

MCWS would still need a way to provide access for a given playlist based on login credentials or key.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 04, 2021, 01:28:21 pm
Like I could create a guest access key, give it to somebody which when they use that access key within their copy of MC it'd connect and prompt me at my end informing me there's a connection attempt, that I can accept, temp allow it for X amount of time, deny, ignore, etc.

That would require you to be awake and in front of the PC/phone to authorize each access. I think sharing/revoking must be handled asynchronously - you share something and it becomes available to someone having the key; and you can revoke access at any time.

Using playlists is nice (in my opinion) because you can add/remove stuff from a shared playlist at any point.
Playlist -> RightClick -> Share... -> enter credentials, auto-expiry date, other options; show sharing URL. The icon should change to reflect that it is now shared. After this one-time setup, all you need to do is add/remove files to the playlist.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 04, 2021, 01:35:02 pm
I think it needs to be kept simple.

It seems to me that if you have a single playlist or smartlist named "Public" it will be shared with an optional RO password (separate from the RO MC client password) that would be the best compromise for an initial release.

Additional functionality could always be added later.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 04, 2021, 02:08:52 pm
I'm pretty excited!  The basics work!!!!
- view/play in a web browser
- Allow for a download (for when the bandwidth is not enough)

Some things:
- MC does need to restrict access to just to the content in the particular link
- I don't want or need a Usr/Pass, just the access to a particular Item in the Library by a random Generated Token
- Needs to be easy to mark an item to be shared, generate a link, copy it to the clipboard, and keep track of the # of plays/downloads.
- Needs to be easy to then remove the link to stop access

More Advanced Stuff:
- Sometime I share a "Folder" of items, eg Family Home Videos where the link is to the group, and they can see a thumnail of all the content, be able to navigate to particular items, play, download etc...
- There is also a "Download All" or "Play All" option

:) :) Love it!

Thanks
Nathan
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 04, 2021, 02:10:26 pm
Nice Pond!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 04, 2021, 02:12:30 pm
OK, changed my access key and turned off read-only!  Thanks for looking everyone.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 04, 2021, 02:21:28 pm
I did not snoop!  ... but good idea.

Also, you may need an set of Conversion Options (like with MCWS) given there will be different bandwidth environments
- Video (For Playback) --> Auto FPS / Safe Web Playable / 25Mbps
- Video (For Download) --> No Conversion
etc
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 04, 2021, 02:33:52 pm
I think multiple libraries is problematic - updating the file tags in the main library would not update in the other libs, so this would require too much management overhead.
I don't think we care much about that once a file has been shared.  You could always share it again if you did care.
Quote
What about sharing playlists instead of files? We could have different playlists for different people, different credentials/key per playlist, etc. Seems more practical than distinct libraries. This could work for any generic playlist, or you could add a special Playlists\Shared section in the tree. Playlist Groups could also be used to share a group of playlists to the same person.
I can imagine situations where you might want to share different sets of files with different people.  Grandma gets the cleaned up version without all the empty beer bottles, for example.

I might want to share single files (a movie) or playlists (trip pictures).  We need to handle both.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 04, 2021, 03:10:13 pm
I meant that we would just create a new playlist for each person that we want to share things with. Then we would just drag stuff into it, be that just one file or 100 files.
These playlists could be under a "Shared Playlists" group, to clearly separate them from the other regular ones. This would make management trivial, using all the already existing playlist functionality.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 04, 2021, 06:47:13 pm
A single playlist works for the first set, but what happens if you want to share files on a different subject?  Trip 1, 2 ,3, ...  I'd like to have multiple sets I share with multiple people.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 05, 2021, 04:41:22 am
Then share at the PlayList Group level; all playlists inside would be shared according to the Group's sharing settings.
The sharing settings could exist at the Group level only, or alternatively they could be inherited and overridden at the playlist level too.

To support multiple people, either create a group or playlist for each one (with different account/key for each), or provide options to add multiple accounts/keys to each shared playlist/group.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: marko on February 05, 2021, 12:24:16 pm
Bo-oooooo-b....

Expanding upon this...

If this single-file-sharing becomes a thing, what are the chances of having JRemote for IOS hook into the iPhone sharing system?

I'm out and about most of the day with JRemote playing some choice random smartlists (one usually does me all week) and you know when a track arrives and it instantly takes you back to that great gig, and you want to share it with the folks that were there with you that night...

If JRemote had a "Share" icon there, that generated the required link, it could be sent via whatsapp, email text or whatever, to whoever... This would be brilliant.

The memories these tracks spark are even more so poignant in these locked down times we currently live in.

Do-able?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 05, 2021, 03:03:49 pm
Tonight's build can provide a sharing URL.  Authentication is not needed for that function any longer.

Let us know!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 03:31:09 pm
Quote
7. NEW: Added the right-click command Get Sharing URL... to get a URL from MCWS to share externally (it uses your external IP address, so you will need to set port forwarding on your router).

Thanks!  Here is what I've found in a quick test
- It seems to just use the FileKey instead of a token, so it opens up your entire library to be randomly downloaded! - I can manually plug in any FileKey and the link has access.
- It seems to play many file types in the Web Page just fine (only tested some Video and Audio), but file type that are not supported seem to be automatically downloaded to your PC instead of bringing up the Web IF.  So MP4 works (as expected) but other Video File Types seem to trigger the autodownload.  This is where I think it will make sense for an option that MC use (a selectable) Transcode profile to get a web safe stream working.
- This Transcode option would also make compound content (DVD, BD's) etc work, as at present you just get the index file downloaded.

So as step 1, It Works!  and Works Well! ... but I'd suggest
Step 2: Change the access method as else your entire library is open to anyone by just using any old FileKey in the URL.  Seems like a massive security hole in this version!
Step X: Refine the other things....
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 05, 2021, 03:36:48 pm
I suppose we could base it on a user enterable share ID?  Maybe it uses a string of random characters if it's empty?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 05, 2021, 03:39:37 pm
I proposed a secure solution above, using JSON.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 03:51:28 pm
I don't propose to know the right way of doing this (zybex could be correct), but from a users POV:
- MC Needs to generate a random token
- This should be stored in a Database field (so you can see what it is and re-use it)
- This needs to be clearable (so to stop sharing)
- Another new DB Field (Number Plays - External) should record the # of play using the external link so you can See when someone has played it.  This would also let you see what media has been accessed with a smart list.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 05, 2021, 03:54:18 pm
Maybe we require auth for the current get file call but make a new share call that doesn't that takes a user entered field.  If you didn't enter that field value, nothing would be shared.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 03:54:45 pm
Quote
6. Changed: MCWS no longer requires authentication for functions marked to not require authentication.

This also scares me.  What functions don't require authorisation?  For example, does that mean if I have Matt's external MC IP address I can now do all sorts of stuff remotely without authorisation?  Even simple stuff like start MC Playing content at 2am?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 05, 2021, 03:58:54 pm
I don't propose to know the right way of doing this (zybex could be correct), but from a users POV:
- MC Needs to generate a random token
- This should be stored in a Database field (so you can see what it is and re-use it)
- This needs to be clearable (so to stop sharing)

I understand jmone's sentiment, but I think this is an overcomplicated way of stating what the user's POV requires.

What they actually require is:
1. MC should share those files the user has explicitly asked it to share, and ONLY those files.
2. They need to be able to see what is shared
3. They need to be able to see the URL for access, so that they can give it to others
4. The URL from one copy of MC shouldn't work on another (URLs are unique or non-transferable)
5. They need to be able to stop sharing

Tokens/fields/etc don't enter into their actual needs, so how MC does this behind the scenes shouldn't impact the user.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 04:04:37 pm
I really think it needs a separate token method rather than using the filekey to gain access, otherwise this loop will give access to anyone to all items marked as shared in no time at all:

Loop
 http://xxx.xxx.xxx.xxx:xxx/MCWS/v1/File/GetFile?File=%Index%&FileType=Key

edit - updated my wording as I agree with wer it is up to JR on the "How"


Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 05, 2021, 04:09:31 pm
MC just needs to encrypt the FileKey in the URL using an encryption key generated by the current server/install. If the user changes anything on the encrypted string, it won't decode anymore.

From the user PoV, it's clear - a simple right-click->Share is all that is needed (though I think it's useful to share playlists as well as individual files).
What we're discussing is implementation.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 04:12:31 pm
That would work (as long as the std GetFile then went back to requiring Authorisation).
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 05, 2021, 04:33:27 pm
The problem is that opening up GetFile let's you snoop.  If we close that but add a new share function, it could only share what you've told it to.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 05, 2021, 04:38:33 pm
Using an encrypted file key might be clever because then you wouldn't have to enter anything.  It would have everything open for sharing but it would be impossible to guess a key.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 04:43:20 pm
Works for me!  So are you thinking something like:
- GetFile:  Goes back to using Authentication
- GetFileEnc: No Authentication but uses an Encrypted Key
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 05, 2021, 04:46:56 pm
I proposed above using a new /Share URL endpoint, I think that would be safer and allow for future expansion.
The Key encryption scheme could be something like this (https://gchq.github.io/CyberChef/#recipe=RC4_Drop(%7B'option':'UTF8','string':'MCPrivateKey'%7D,'Latin1','Latin1',768)To_Base64('A-Za-z0-9-_')&input=e0ZpbGU6MjQ1MTQ2MjUsIEV4cGlyeToxNjEyNTU4NjQ1fQ) (RC4+Base64) - this was what I used above too. The "output" is the token to be included in the URL, the "input" is the decoded data that points to the shared filekey.

There's no way to guess any valid token by changing the encoded data, it would break the RC4. You can use AES instead if you prefer, or anything else really.

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 05, 2021, 04:51:24 pm
While discussing how all this could work...I also agree that being able to share a playlist or group of items makes sense and is easier eg:
Home Videos (a link for the familly)
Our Trip to Someplace (a link to those we went on the trip to)
etc etc
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 05, 2021, 06:26:45 pm
What they actually require is:
1. MC should share those files the user has explicitly asked it to share, and ONLY those files.
2. They need to be able to see what is shared
3. They need to be able to see the URL for access, so that they can give it to others
4. The URL from one copy of MC shouldn't work on another (URLs are unique or non-transferable)
5. They need to be able to stop sharing


I agree with those requirements. And I think its important to note that these should be the absolute minimum requirements to have control over sharing, and not a distant goal. Anything less then that should not even be made available as a half-baked version can compromise your data.

Once we have a safe way to share single files, then we can think about sharing playlists or other logical structures, so I would suggest to leave playlists etc out of the discussion until the fundamentals are solved.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: lepa on February 06, 2021, 01:15:59 am
What wer and Hendrik said above
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 06, 2021, 06:54:12 am
Matt,
Just use "Share" instead of "Get URL ..."

Generate two URL's.  One for inside the network.  One for outside.

It has to work for a set of files.

Security first.

Google Drive's sharing would be worth looking at for ideas.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: hoyt on February 07, 2021, 01:09:37 pm
This is cool.  I really like this idea and am happy to see it getting movement.  Perhaps my situation is different, but when I do generate link it makes the IP address and internal port, which is different than what I want/ can share.  I have my MC server behind a proxy server, serving any external link via HTTPS that way.  So I get:

http://external_ip:52199/MCWS/v1/

But what I need is:

https://external_fqdn:52200/MCWS/v1/

I might be the only one doing this, but I don't have MC managing my fqdn keys because I didn't want to paste the new keys every 90 days.  This way my webserver kicks off certbot-renew every X days and gets the new certs when they expire.  That webserver then manages my external requests.  It would be nice to supply those data points to MC in the prefs "when sharing make my external address be X and my external port be X, and make them be HTTPS Yes/No."  Like I said, I might be the odd-man out here, but as long as MC would make a shareable link, aka Google Drive, then I think that's very cool.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 08, 2021, 03:45:36 pm
Tonight's build does a lot of work here.

Encrypted the URL so guessing isn't possible anymore based on the suggestions above.

Renamed the feature "Share" based on Jim's suggestion.

Made a way to reset your share key based on feedback from Hendrik.

Thanks for all the help.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 08, 2021, 04:18:57 pm
Looking good:
- New Key method works
- Invalidating the Key works

Looks like (for Video) only MP4 plays in Web Browser, the other types are not shown in the Web Browser but are download automatically.  So (for video):
- Would using Hendrik's new MCWS API for Video Streaming like that used for JRemote2 be a better idea?  I expect this should then work with all video (including compound videos like DVD / BD etc) on any device regardless of Formats, Bitrates etc (and the Bitrate should be better for streaming)?
- If the above is added then a "nice" to have would be for the Download Button option to allow for either the converted stream or original but.... I'd just be happy with the above.

...or should the "share" MCWS call be a subset of the existing GetFile ? 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 08, 2021, 04:27:04 pm
I removed a couple  of posts.  Please put requests for other features on the MC27 board.  This isn't a private request board.

jmone gets a pass this time.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: leezer3 on February 08, 2021, 05:03:52 pm
This may be a thought too far:
Perhaps consider 'hiding' the fact that our share key is MCWS at all?

Separate on port or something?

The first draft showed that its possible for a malicious actor to manipulate the MCWS call.
Expose as little attack surface as possible :)

(Or possibly being paranoid)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 08, 2021, 06:08:03 pm
Not paranoid.

Matt,
It works for me.  Thanks.

I'd like to be able to decide which address to copy.  I also don't think there's any point in displaying it.  Maybe the IP, but nothing else.

A window that had two buttons would work:

Please choose which link to
copy to your clipboard.

[Local]   or   [Internet]


Or maybe someone else has a better idea.

It's great to have the help.  Thanks.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 08, 2021, 06:22:10 pm
How useful is the local sharing?
Wouldn't the internet link just work in the occasion you need something local? Most routers would be smart enough to just route that request internally in your network.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 08, 2021, 07:37:02 pm
Most routers would be smart enough to just route that request internally in your network.

Yes.  I Tested a 150mbs Video using the external link and while my router traffic reported it on both the Upload and Download the traffic was just doing a U turn and not hitting the Modem (my up link from the modem is 50bps so such a stream would gag).
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 09, 2021, 03:21:28 am
Cool :) Encryption looks good. Just a note: please ensure that the encryption key is unique for each PC or MC install. This should not be the same as the Access Key... maybe you can generate a random key on first time use.

Two details to fix:

1. The Base64 string is broken by CRLF every 64 chars - 2 lines are actually copied into the Clipboard

2. The Base64 string is using the default mapping with fwd-slash, Equal and Plus characters, which make up a slightly invalid URL (still works, but not ideal). The Base64 string should be URL-Encoded (%2F) or use a Base64 character mapping without those chars.
 
Regarding the local vs remote URL... I agree that the remote one is what makes sense.
What about adding a setting/RegistryKey to add an FQDN? If it's blank, then MC would use the public IP; if it's defined then it would use that instead.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 09, 2021, 06:18:30 am
How useful is the local sharing?
Wouldn't the internet link just work in the occasion you need something local?
Testing.  Large network like a university or a business.
Quote
Most routers would be smart enough to just route that request internally in your network.
I didn't know that.  Is it safe to assume it's 100%?  I'd rather not find out the hard way.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 09, 2021, 06:22:26 am
Regarding the local vs remote URL... I agree that the remote one is what makes sense.
What about adding a setting/RegistryKey to add an FQDN? If it's blank, then MC would use the public IP; if it's defined then it would use that instead.
Interesting idea.  Do we know the FQDN?  If the user has to set it, we end up doing network support.  Local vs Internet is clear and simple to explain.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 09, 2021, 06:31:39 am
Testing.  Large network like a university or a business.I didn't know that.  Is it safe to assume it's 100%?  I'd rather not find out the hard way.

It works only if you add the port-forward entry for MC on the router, so that it knows which device to forward the connection to. So in effect, you still need to have an open port accessible from outside. Also, not all routers do this U-turn (though most do)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 09, 2021, 06:39:42 am
Interesting idea.  Do we know the FQDN?  If the user has to set it, we end up doing network support.  Local vs Internet is clear and simple to explain.

No - MC could do a reverse DNS lookup to find the FQDN but that won't work for dynamic domains like "myhome.dyndns.org".
The user would need to somehow give it to MC - if one is knowledgeable enough to have a DynDNS, one knows how to use it (I hope). The alternative is to have to edit all MC Sharing links to replace the IP with the FQDN.

The thing with external IPs is that they change, so the links become invalid. Only companies have fixed IP addresses. Having the option to use a dyndns domain is nice; I bought a domain name with dynamic IP support and use it to reach my home services.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 09, 2021, 06:43:46 am
Cool :) Encryption looks good. Just a note: please ensure that the encryption key is unique for each PC or MC install. This should not be the same as the Access Key... maybe you can generate a random key on first time use.

Thats how it should work. I also asked Matt to make a button to reset it, I think that made it into the build as well.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 09, 2021, 07:22:51 am
The thing with external IPs is that they change, so the links become invalid. Only companies have fixed IP addresses. Having the option to use a dyndns domain is nice; I bought a domain name with dynamic IP support and use it to reach my home services.
Our Access Key sever keeps track of the outside address.  We use that server for this new sharing feature.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 09, 2021, 07:52:40 am
You can't use that, really... the generated Sharing URL is a public address which does not link in any way to JRiver's access keys. It's either an IP address or a domain name.

To use your AccessKey server, the URLs would need to be something like "http://sharing.jriver.com/blablabla?File=EncryptedData"; the encrypted data would need to contain the target AccessKey, and your server would then need to redirect the HTTP request to the actual IP of the user. All accesses would have to go through your server.

Unless you want to provide a Dynamic DNS Server service to all your customers ;) With domain names like "accesskey.mc.jriver.com", which would resolve to the user's IP.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: hoyt on February 09, 2021, 08:43:14 am
No - MC could do a reverse DNS lookup to find the FQDN but that won't work for dynamic domains like "myhome.dyndns.org".
The user would need to somehow give it to MC - if one is knowledgeable enough to have a DynDNS, one knows how to use it (I hope). The alternative is to have to edit all MC Sharing links to replace the IP with the FQDN.

The thing with external IPs is that they change, so the links become invalid. Only companies have fixed IP addresses. Having the option to use a dyndns domain is nice; I bought a domain name with dynamic IP support and use it to reach my home services.

Yeah, that's how my setup is.  I use https://home.mydomain.com:52200 to get to MC.  My router forwards 52200 to my webserver at 192.168.x.x.  That webserver then listens on 52200 and checks the SSL certs and proxies the info to my jriver instance at 192.168.2.103.

Code: [Select]
<VirtualHost *:52200>
    ServerName home.mydomain.com

    Include /usr/local/apache2/conf/options-ssl-apache.sslconf
    SSLCertificateFile /usr/local/letsencrypt/live/fullchain.pem
    SSLCertificateKeyFile /usr/local/letsencrypt/live/privkey.pem

    ProxyRequests Off
    ProxyPass / http://192.168.2.103:52199/
    ProxyPassReverse / http://192.168.2.103:52199/
    ProxyPreserveHost on

</VirtualHost>

This works well, but it means MC has no idea what port to use and doesn't know my domain name.  MC doesn't even have my SSL keys, so you couldn't determine it that way either.  I would really need to supply these items to MC as a setting.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 09, 2021, 08:48:33 am
There's an option to "Enable SSL" under Settings->Media Network, using port 52200. You can provide your certificate as well. So I think you can skip your proxy server.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 09, 2021, 08:52:33 am
You can't use that, really... the generated Sharing URL is a public address which does not link in any way to JRiver's access keys. It's either an IP address or a domain name.
No, you don't use the Access Key, but yes, we use our Access Key server to get your current IP address.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: hoyt on February 09, 2021, 08:52:51 am
There's an option to "Enable SSL" under Settings->Media Network, using port 52200. You can provide your certificate as well. So I think you can skip your proxy server.

But then I have to copy my cert files every 90 days to my MC server and re-include them.  This way, Letsencrypt takes care of it begin-to-end automatically.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 09, 2021, 08:55:17 am
No, you don't use the Access Key, but yes, we use our Access Key server to get your current IP address.

Ah, OK. I misunderstood. Still, we can't use it as a general purpose dynDNS, so it would be nice to allow for that option.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: hoyt on February 09, 2021, 11:20:40 am
I tested out expiring the links and that worked.  Good idea.  Does that mean that I can somehow generate a report in MC of all the links I've created?  Anyway to manually manage that, ie - expire one link, but not all?  Or when sharing, set a default expiration?  OneDrive does this well.  When you go to make a sharable link, you can set an automatic expiration after X days.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 09, 2021, 02:15:25 pm
FYI, I have both a static IP and also a DYNDNS entry simply to keep a simple to remember FQDN, but I found that while:
- http://123.123.123.123:52199/MCWS/v1/Share/Get?File=abcdef (works)
- http://mine.dyndns.org:52199/MCWS/v1/Share/Get?File=abcdef (does not - I get a "the server unexpectedly closed the connection)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 09, 2021, 04:02:06 pm
Quote
1. Changed: Added URL escapement to the share URLs.
4. Changed: When sharing, a choice is offered about what link to place on the clipboard.

Both work fine.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 09, 2021, 05:33:23 pm
Looks like (for Video) only MP4 plays in Web Browser, the other types are not shown in the Web Browser but are download automatically.  So (for video):
- Would using Hendrik's new MCWS API for Video Streaming like that used for JRemote2 be a better idea?  I expect this should then work with all video (including compound videos like DVD / BD etc) on any device regardless of Formats, Bitrates etc (and the Bitrate should be better for streaming)?
- If the above is added then a "nice" to have would be for the Download Button option to allow for either the converted stream or original but.... I'd just be happy with the above.

That is not really related. If you want to be able to watch every video in-browser properly, it would likely have to produce some kind of website with an actual video player component on it, similar to Panels player page.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 09, 2021, 06:06:21 pm
That is not really related. If you want to be able to watch every video in-browser properly, it would likely have to produce some kind of website with an actual video player component on it, similar to Panels player page.
Couldn't the server convert?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 09, 2021, 07:03:12 pm
That is not really related. If you want to be able to watch every video in-browser properly, it would likely have to produce some kind of website with an actual video player component on it, similar to Panels player page.

...but you already have that new high quality AutoFPS MCWS call that will produce a webplayable stream that is used by JRemote2 etc.  Only about 20% of my Videos are MP4, so the other 80% will not play with the current method, but 100% will if using your new MCWS call.  Without on-the-fly conversion then I'm not sure this public URL feature is all that useful (to me), unless I make a MP4 versions of everything I want to share first (which is an option I guess). 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 09, 2021, 08:08:06 pm
Let me take a step back from the details to the original Idea (aka MC Youtube).  #1 It has to be simple, and #2 it has to always work
- Example 1: I've just shot a video of the family at Christmas and I want to share a link of that one item with the Family. 
- Example 2: I was then asked by my brother stuck in lockdown in the UK for some really old video from when we lived in the US (about half a dozen different videos) so I want to share a link to that set of videos

The link(s) just needs to work regardless of
- What Container & Codecs they are in MC,
- What Device they are watching it on (Phone, PC, etc), and
- What Bandwidth is available for the stream to play reliably.

Suggestion on the specifics:  I think (for most users) it would work best if they did not have to manually create a "streamable version" but for MC to do this on the fly as needed, based on a profile like we have now with for both
- "Client Options (when connected to a Library Server)" and with
- "DNLA" Clients

.... so something like this (but I don't profess to know the details of what makes sense for the profiles)


 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 09, 2021, 08:20:45 pm
That is not really related. If you want to be able to watch every video in-browser properly, it would likely have to produce some kind of website with an actual video player component on it, similar to Panels player page.

You may be right, especially if you want an end user to be able to "Browse" a selection of shared content.  I would have thought a link to a single video would be fine however (or am I missing something)?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: hoyt on February 09, 2021, 09:13:41 pm
That is not really related. If you want to be able to watch every video in-browser properly, it would likely have to produce some kind of website with an actual video player component on it, similar to Panels player page.

Why not use Panel for this?  Maybe there's s good reason, but that would certainly make sense for the "next" of a playlist.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 10, 2021, 02:43:09 am
...but you already have that new high quality AutoFPS MCWS call that will produce a webplayable stream that is used by JRemote2 etc.  Only about 20% of my Videos are MP4, so the other 80% will not play with the current method, but 100% will if using your new MCWS call.  Without on-the-fly conversion then I'm not sure this public URL feature is all that useful (to me), unless I make a MP4 versions of everything I want to share first (which is an option I guess).

The new API does not change any of that. You could do the same with the old one. Its entirely tangential to any sharing here.
If you want video to play properly in a browser, it needs to be wrapped in some kind of website. Thats maybe something Brad can work with Matt, as he did that for Panel.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 10, 2021, 04:04:53 am
Sorry, I'm clearly missing a key point in my understanding (not to hard for me to get confused!).

Couldn't the single shared link present a websafe transcoded steam instead of the original file?  And if so, would it not make sense to use your new improved code?

We're working on a set of video streaming improvements for MCWS and JRemote2 for Android.  They will arrive later this month.

All these changes are designed for cases when streaming video to the remote device, ......
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 10, 2021, 04:28:01 am
You keep asking the same question and i have no new answers. The new API is entirely unrelated to any questions in this thread. If you want universal video playback, it needs to return an actual website that wraps the stream in soem fashion. Browsers are terrible video players, thats why they get fancy Javascript components to help them along.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 10, 2021, 04:41:55 am
@Jmone: Why transcode something, forcing the user the have a [relatively] powerful server and suffer quality loss, plus noise and heat on the server side, when you can have direct playback with Hendrik's solution: a Player component on the webpage that streams the original file and just plays it, with support for most formats. I hate transcoding of any kind.

The Sharing mechanism is the first step. With that in place JRiver can build on it and add the player webpage and other features.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 10, 2021, 05:21:15 am
Apologies Hendrik.  I'm not trying to be difficult but I really really don't understand.  I must be missing something fundamental, that I am trivialising something, or that I'm asking the wrong question.  I'll have one more crack at trying to understand, then I'll shut up.

From what I can tell:  Matt's Share Link streams out an MP4 Video that plays in a Web Browser.  It works.  No fancy Jacascript or Website required.  While is simple, elegant & efficient, it is not universal as it appears the video must be an MP4 and that the bitrate of the original MP4 may be too high to stream out.  If that is correct;

The bit I don't understand is: what is the issue with Matt's share link using an on the fly transcoded stream instead of the original?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 10, 2021, 05:23:37 am
@Zybex.  If the bitrate of your video is higher than your bandwidth you will need to transcode for real time playback.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 10, 2021, 07:01:03 am
jmone,
I sent this to Hendrik earlier:

"Right now, we can convert on-the-fly for Gizmo and JRemote2.  I know the server has to do it.  Why could we not have a URL that signaled MC that conversion was needed?"

Hendrik, please reply here if you want to.

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 10, 2021, 07:40:02 am
There is no good on the fly conversion format that a browser can simply play out of the box, like mentioned above, Browsers are actually pretty bad video players. Thats why you need a JavaScript web player around it like we have in Panel that manages the playback, ie. a website that embeds the player.

Thus, there is no URL we can generate that "just plays" right now without several deficits. The streaming conversion is focused on actual players on the users end, like it exists inside JRemote2, or even inside Panel, but its not something a browser can just play without help.

This doesn't mean it can't be done, it just requires a bunch more work. Thats why I suggested involving Brad above, since he made this for Panel and has experience with providing a web site through MC.
If we intend the share feature to primarily be for something you are supposed to play, and not download, then a player page that comes up might make sense for all media types, as it avoids hoping that the browser does the right thing even with eg. audio.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 10, 2021, 01:51:03 pm
Thanks Hendrik. 

[backinbox] This was the bit I did not understand: "there is no URL we can generate that "just plays" right now without several deficits".  I'd just "presumed" that if a static MP4 would play, then an on the fly one would as well, and that the new HLS default profile was the ticket to do so.  Sorry for the Angst in getting it through me thick head. [/backinbox]

I also agree, that exposing the MC Web Server would be a much more capable and scalable soln.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 10, 2021, 02:43:11 pm
The other thing I use a web server for is to host pics to be used to embed IMG links in forums (especially when you can't attach).  Lets see if the share links works for this.

Image imbedded between the lines using "Insert Image Link" option on the forum sw
--------------------------
(http://144.137.208.82:52199/MCWS/v1/Share/Get?File=8dli%2FaQTYeT%2BzDMXBo9ZKqjWkPbDN4DhZISpDjK2moSPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D)
--------------------------
edit - Nope.  just above is the following wrapped in the img tag (img)(/img) (but with square brackets) but nothing appears

http://144.137.208.82:52199/MCWS/v1/Share/Get?File=mwZDhP%2BGhvcquLv5kBISJNzON2X5ohUoz0Ll1b2a9emPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 10, 2021, 03:25:10 pm
It doesn't show your image because its http (not https), and mixed content is no longer allowed in any modern browsers.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 10, 2021, 05:13:30 pm
The other thing I use a web server for is to host pics to be used to embed IMG links in forums (especially when you can't attach).  Lets see if the share links works for this.

Image imbedded between the lines using "Insert Image Link" option on the forum sw
--------------------------
(http://144.137.208.82:52199/MCWS/v1/Share/Get?File=8dli%2FaQTYeT%2BzDMXBo9ZKqjWkPbDN4DhZISpDjK2moSPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D)
--------------------------
edit - Nope.  just above is the following wrapped in the img tag (img)(/img) (but with square brackets) but nothing appears

http://144.137.208.82:52199/MCWS/v1/Share/Get?File=mwZDhP%2BGhvcquLv5kBISJNzON2X5ohUoz0Ll1b2a9emPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D
Maybe because it's not an image.  Try making it a simple link.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 09:43:33 am
What do you think of adding 2 fields:

[Number Plays (Sharing)] or [Shared Plays] - increases when a share URL is accessed ([Number Plays] should also increase)

[Shared] - Bool. When user generates the Sharing URL, this field gets checked. If the user unchecks it the file cannot be played using the Sharing URL. This allows:
- un-share a single file instead of having to invalidate all shares
- temporary un-sharing
- easy list/filter for currently shared files

When a Share URL is accessed, MC just needs to check if this field is checked. Clearing the ShareKey could uncheck this field on all files automatically.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Matt on February 11, 2021, 10:00:39 am
[Shared] - Bool. When user generates the Sharing URL, this field gets checked. If the user unchecks it the file cannot be played using the Sharing URL. This allows:
- un-share a single file instead of having to invalidate all shares
- temporary un-sharing
- easy list/filter for currently shared files

I like this idea.  I'll work on it.  Thanks!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: hoyt on February 11, 2021, 12:18:57 pm
[Shared] - Bool. When user generates the Sharing URL, this field gets checked. If the user unchecks it the file cannot be played using the Sharing URL. This allows:
- un-share a single file instead of having to invalidate all shares
- temporary un-sharing
- easy list/filter for currently shared files

That would accomplish what I was thinking here fairly elegantly:

I tested out expiring the links and that worked.  Good idea.  Does that mean that I can somehow generate a report in MC of all the links I've created?  Anyway to manually manage that, ie - expire one link, but not all?  Or when sharing, set a default expiration?  OneDrive does this well.  When you go to make a sharable link, you can set an automatic expiration after X days.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 01:37:10 pm
That would accomplish what I was thinking here fairly elegantly:
Quote
I tested out expiring the links and that worked.  Good idea.  Does that mean that I can somehow generate a report in MC of all the links I've created?  Anyway to manually manage that, ie - expire one link, but not all?  Or when sharing, set a default expiration?  OneDrive does this well.  When you go to make a sharable link, you can set an automatic expiration after X days.

I did propose the [Shared] field and an Expiration mechanism already on post #13 ... I need to learn how to write more concise posts so that people actually read them ;)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 02:21:35 pm
Nice - +1 for the Check box and a count field (so you have some indication that the file has been accessed or not).
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: marko on February 11, 2021, 02:36:42 pm
The generated link is super ugly... Would it be hard to format it, say, with the file name?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 02:54:20 pm
The URL needs to contain the FileKey in a secure way... adding more info is possible, but only makes it longer.

Here are 2 alternatives for prettier URLs:

1. Youtube-like short identifiers: these would be random generated, stored in a [ShareKey] field of each shared file:
Code: [Select]
http://my.mchome.net:52199/Share?File=ZaV5zK3P1XeCollisions would need to be prevented. Security would come from the randomness of the identifier, hard to guess a valid one.

2. [sort of] Human readable URLs with a secure Signature for validation:
Code: [Select]
http://my.mchome.net:52199/Share?File=123456&Name=The%20Croods%3A%20A%20New%20Age&Sig=5zK3P1XeThe &Name= parameter is entirely cosmetic/optional (can also be added on the first method above). The &File has the actual file Key to share, and the &Sig has an encrypted hash that must match some server-generated hash for this URL/File.

In the end, the URL is not meant to be pretty. You can use some URL-shortener like bit.ly to create nicer URLs if you like.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 02:56:18 pm
It doesn't show your image because its http (not https), and mixed content is no longer allowed in any modern browsers.

Makes Sense.  So is there a way of adding a Cert and generating a HTTPS link that would then work for embedding PICs in a forum like this?

Here is a sample link from my MC instance for testing if anyone has some bright ideas?
http://144.137.208.82:52199/MCWS/v1/Share/Get?File=mwZDhP%2BGhvcquLv5kBISJNzON2X5ohUoz0Ll1b2a9emPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 02:58:47 pm
Media Network -> Enable SSL
I suppose the links will be https:// with this option enabled.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: marko on February 11, 2021, 03:16:46 pm
Why can't that url be the filename, that when clicked goes there... You know what I mean? What is it in html speak... <a href=?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: marko on February 11, 2021, 03:17:53 pm
Media Network -> Enable SSL
I suppose the links will be https:// with this option enabled.
If you use the MC generated certificate, I think most browsers will throw a wobbly over it?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 03:21:01 pm
Yeah - I've tried using the self MC generated SSL but could not get it to work either.... hence I thought I'd throw it out to those that know this stuff way better than me! 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 03:48:59 pm
Yeah - I've tried using the self MC generated SSL but could not get it to work either.... hence I thought I'd throw it out to those that know this stuff way better than me!

You need to provide your own certificate. It's not trivial, you need to buy a domain name, get the certificate for it, and the Sharing URLs need to use that domain instead of the IP address...
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 03:51:15 pm
Why can't that url be the filename, that when clicked goes there... You know what I mean? What is it in html speak... <a href=?
The URL would need to include the full path - just the filename could lead to collisions (same filename in different folders, which one to play?)
Since chars like slash, dots, spaces, symbols, etc, would need to be URL-escaped (%2F...), the URL would still be ugly, and long. Having the FileKey (unique number) in the URL is a shorter way to uniquely point to a given file.

Also, security: Just the filename or Key isn't enough, people could guess other names from your naming schema. So the URL needs to include some encrypted token that the server can verify.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 11, 2021, 04:05:43 pm
You need to provide your own certificate. It's not trivial, you need to buy a domain name, get the certificate for it, and the Sharing URLs need to use that domain instead of the IP address...

Actually you can self-sign a cert for an IP address, without a domain name.  Modern browsers will distrust it by default, but you can add an exception for it.  The process is a little cumbersome, so I would agree it's non-trivial.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 11, 2021, 04:30:03 pm
I meant the solution to have a full trusted certificate, with no browser warnings.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 04:57:39 pm
So ... I've got my own Domain Name and use DYNDNS to point to my IP address.  Currently I have a NextCloud server with a SSL cert from LetsEncrypt on the NextCloud box and this works fine for hosting the images to embed.  I wonder if there is a way of doing the same thing for MC?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 05:01:31 pm
eg, this works to embed : https://behome.dyndns.info/index.php/s/AxQpXKqAZWKD66F/preview

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Hendrik on February 11, 2021, 05:06:22 pm
You can load a Lets Encrypt certificate (or any other) into MC. It should accept standard PEM encoded files for the cert and private key.
If you want to use this on a regular basis, a SSL proxy in front of it is likely easier and more secure, as it lets you control the access more precisely.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 05:13:35 pm
...aaahhhh!  I just got it work or is just just me?

------ Can you see an image between the lines? --------------
(https://144.137.208.82:52200/MCWS/v1/Share/Get?File=mwZDhP%2BGhvcquLv5kBISJNzON2X5ohUoz0Ll1b2a9emPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D)
-------------------------------------------------------------------

Thanks
Nathan

Edit - nope, just me as I must have the security settings in Chrome for this site turned down.....
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 11, 2021, 05:14:42 pm
No
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 11, 2021, 05:15:37 pm
eg, this works to embed : https://behome.dyndns.info/index.php/s/AxQpXKqAZWKD66F/preview
I can see that image.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 07:05:56 pm
So, I've got a bit further:
- Created a new DynDNS name for my IP Address
- Created a SSL certificate for the DynDNS name using "Certify the Web" GUI / Lets Encrypt
- Loaded the SSL certificate and key into MC

Link1: Here is the non-ssl (std Share link):  Works and the image displays pretty well straight away

http://144.137.208.82:52199/MCWS/v1/Share/Get?File=wUgLY6bHiVXo8ImCBhqPU3VNbble%2FmCo%2BRzTRaMN5XyPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

Link2: Here is the same link but changed to use HTTPS, MC SSL Port, & DynDNS:  It eventually loads...... but it takes ages (eg a few minutes) to show the picture. 
https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=wUgLY6bHiVXo8ImCBhqPU3VNbble%2FmCo%2BRzTRaMN5XyPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

EDIT:
- If anyone can test by clicking on the two links and letting me know if they load about the same? 
- Oddly, If I access this link from outside my LAN (eg using my phone over a 4G etc) then it is quick.  It is only deadly slow if my device is connected to my LAN (wired or wifi).  Something weird ....
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 11, 2021, 07:44:59 pm
...and this is the HTTPS/SSL based link embedded using IMG tags
(https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=wUgLY6bHiVXo8ImCBhqPU3VNbble%2FmCo%2BRzTRaMN5XyPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 11, 2021, 10:29:25 pm
The first link works. The second doesn't.   Port forwarding or firewall?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: marko on February 11, 2021, 11:28:12 pm
The URL would need to include the full path - just the filename could lead to collisions (same filename in different folders, which one to play?)
Since chars like slash, dots, spaces, symbols, etc, would need to be URL-escaped (%2F...), the URL would still be ugly, and long. Having the FileKey (unique number) in the URL is a shorter way to uniquely point to a given file.

Also, security: Just the filename or Key isn't enough, people could guess other names from your naming schema. So the URL needs to include some encrypted token that the server can verify.
C'mon zybex, I know I can be a bit slow on the uptake sometimes...? I followed the entire thread from inception. Now I'm at my PC, maybe I can explain better...

How hard would it be to turn this:
http://xxxxxxxxxxxx/MCWS/v1/Share/Get?File=DUHrH6S0n9FJC%2FuHAI4Wno1NfF7AJlmb2HvA0gZBLU84OwC6LXKWOoCmElJnk1lp%0AqQ574bOhsYMEqQhO453YJA%3D%3D

Into Aslan - Chains (http://xxxxxxxxxxxx/MCWS/v1/Share/Get?File=DUHrH6S0n9FJC%2FuHAI4Wno1NfF7AJlmb2HvA0gZBLU84OwC6LXKWOoCmElJnk1lp%0AqQ574bOhsYMEqQhO453YJA%3D%3D) when it's placed on the clipboard. It would make for a much more pallatable paste, less likely to scare those you may be sharing with?

The web page simply calls it "get" which is not all that informative.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: Aardvark on February 12, 2021, 12:03:24 am
Testing Jmone's two links

Link 1 opens almost instantly on desktop via Lan and phone via Lan and phone via 4G

Link 2 takes about 2 minutes + to open on desktop via LAN and phone via LAN but almost instantly on phone via 4G.

Regarding Marko's comment re format of link, would much prefer to see plain language name rather than full link details
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 12:27:59 am
@ Aardvark - that's exactly what I see with the two links.  Unless it is some weird propagation issue with the newly created DDNS link used for "Link 2", I'm stumped to why it would be.... or some odd path it takes when I use my WAN Connection (Australian NBN - Telstra) vs 4G (Telstra). 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 01:38:34 am
The first link works. The second doesn't.   Port forwarding or firewall?
Could be.... but I find if I click on the 2nd link you will notice it is "loading" but it takes several minutes rather than actually failing.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 02:05:33 am
Jim would be proud of me..... "Have you tried rebooting?"  seems to have fixed it  ::)

Nope, spoke too soon - was fast once..... now back to what it was before.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 02:58:24 am
C'mon zybex, I know I can be a bit slow on the uptake sometimes...? I followed the entire thread from inception. Now I'm at my PC, maybe I can explain better...
Oh, I can be pretty dense too as you just pointed out  ;D

Yes, it's possible to put a nice URL link on the Clipboard that will paste as a regular text with clickable URL... on some applications. It's called an HTTP Fragment.
However, such link will only paste correctly on some apps that recognize the fragment: Word, Outlook and other Microsoft apps work fine; GMail also works; but WhatsApp and almost all other messaging apps, and also most webpages (like this one) will only paste the non-clickable text without the URL. So it's a hit and miss thing.

Coding details for the needed format are here, if Matt wants to try it out:
https://stackoverflow.com/a/4593519

So eventually we will need a Sharing dialog for all the options :) After clicking "Share" the user would get some dialog with options:
- internal vs external URL
- simple text URL or HTTP fragment
- expiry options
- ... other future options
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 03:09:32 am
The HTTPS delay is not related to the dyndns - ping/nslookup of your hostname translates the hostname into your IP pretty much immediately. So the connection is reaching your router, and I assume it's reaching your MC server a few ms later. Once the dyndns is resolved into an IP, there is absolutely no difference at the TCP level between using the IP or the dyndns.

What is probably happening is a delay on MC side after receiving the connection request.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 12, 2021, 03:38:55 am
You can get a difference in the server behavior between the two links because of the Host field in the http header. The TCP looks the same, but the server sees in the http header whether the browser was trying the ip address, or the FQDN.  It might respond differently (or process differently internally) depending. That's also how virtual servers work. It might be related to the difference you're seeing.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 04:09:12 am
I just added a Let'sEncrypt cert to my MC - it seems to work fine, same speed with HTTP or HTTPS, with IP or hostname.

Here's an MP4 over HTTPS+DynDNS: [link removed]

Here's an image over HTTPS+DynDNS: [link removed]

I think it's a bad idea to use this to share pictures and screenshots to public forums such as this one. How long will the screenshot be shared? What if you change Sharing key (or computer) and invalidate all shares? It's nice to do a search on the forum and see a post from a few years back with images. That only works is the images are directly attached to the post, or hosted on a long-term stable server. If I make a post with some screenshot hosted by MC... it won't be up for long. That will likely make the post useless.

Having a NAS at home with a webserver hosting those files is likely to last longer. Using MC for it, for now, is not reliable in the long term.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 04:29:01 am
Thanks, your SSL link started up just fine (apart from buffering - you need a bigger pipe .... or on the fly transcoding!!! ;) ) 

Does my Link2 open up for you OK or do you get a 3min wait?

I'm really stuck trouble shooting this one:
- The links never fail to play, but
- The SSL Link will take 3 min to start from some connections (eg LAN, but also Aardvark's LAN) but not others (eg 4G for both Aardvark and myself).  I've tested with both Chrome (desktop and mobile), Edge (desktop) and the behaviour is the same.
- Also played with Turning On/Off firewalls etc but I doubt they would introduce a delay Vs just blocking it.

Thanks
Nathan

PS - On the Longevity, I've already been bitten having to renew my pubic shared links when commercial services (drop box, imgur, etc) deprecated the service and also each time I had to rebuild my NextCloud server.  It would be no different with MC.  Thankfully more sites let you upload the imgs than before.  I'm also very keen to have one less server to maintain.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 04:36:33 am
OK - just got a mate to test at his house and exactly the same thing as Ardvark and Myself
- No issues on 4G for either link
- No issues with the Non-SSL link on Wifi
- Delay when using the SSL link on Wifi

I'm very confused why the end user connection type makes a difference. 

Do any of these calls go via JRiver's servers in the US?

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 04:48:11 am
Thanks, your SSL link started up just fine (apart from buffering - you need a bigger pipe .... or on the fly transcoding!!! ;) ) 
I don't think so, looks like MC is really slow doing this. I have 500/25 Mbps. Your first link/image above also takes 5 seconds to show up completely, and that's just 100KB - Chrome draws it in chunks as it arrives. Then it takes another 5 seconds to receive the favicon, which is about the same size. See timeline below - note that it's the download itself that takes a long time, so it's MC that is uploading veeeeeery slowly.

Quote
Does my Link2 open up for you OK or do you get a 3min wait?
2.7 minutes. The image also appears in chunks during the last 5 seconds - the rest of the time Chrome is just waiting, no data is arriving.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 04:56:24 am
I see the same delays when accessing your DynDNS link using my cell phone over 4G or Wifi.
It is weird. Maybe some security check on your router? Or even your ISP? But that doesn't explain why it works fine on your nextcloud...
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 05:03:07 am
That is what I'm also seeing with the delay.  Connection is made, nothing happens then at around the 3min mark all the data arrives. It does not matter if it is pic or video.  Same behaviour.  ... but I don't still don't get why the delay is only with the SSL link and only when on the wired connection and not on 4G. 

PS
- Here is a link from my nextcloud server that never has issues (same router, WAN IP, but it is on a different subnet, port and firewall rules) https://behome.dyndns.info/index.php/s/AxQpXKqAZWKD66F/preview
- I've a 1000/50 but I'm on the other side of the world
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 05:14:02 am
I see the same delays when accessing your DynDNS link using my cell phone over 4G or Wifi.
It is weird. Maybe some security check on your router? Or even your ISP? But that doesn't explain why it works fine on your nextcloud...

Both the SSL and Non-SSL connections are coming in over the same FTTP Link, through the same router, to the same PC.  All the non-SSL connections are fine.  Telstra 4G connections are fine.  It seems the Non Telstra 4G Connections get a 3min delay but only for the SSL link.

It is going to be something obvious.... once we work it out that is.  I wonder if there is anything in the MC Logs. 

Anyway, thanks for the help so far!  Off to read a book and sleep on it.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: lepa on February 12, 2021, 05:19:18 am
I don't think so, looks like MC is really slow doing this. I have 500/25 Mbps. Your first link/image above also takes 5 seconds to show up completely, and that's just 100KB - Chrome draws it in chunks as it arrives. Then it takes another 5 seconds to receive the favicon, which is about the same size. See timeline below - note that it's the download itself that takes a long time, so it's MC that is uploading veeeeeery slowly.
2.7 minutes. The image also appears in chunks during the last 5 seconds - the rest of the time Chrome is just waiting, no data is arriving.
In general that seems to be my experience also when I tested from cable to mobile. According to Windows Task Manager ethernet seemed to send data @10mbps when sending a file from MC to mobile. Trying same with other application I was able to send same data ~50mbps. I have ~800/100 mbps connection

I have noticed same with between MC server - MC client over internet so watching movies needs to be transcoded even though I have the bandwitdh for the video bitrate (tested by downloading same movie from home using different SW)

E: zybex link (~5mpbs) played fine for me and with reasonable buffering time when I tried to see about 20 s of it
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 05:29:03 am
(same router, WAN IP, but it is on a different subnet, port and firewall rules)

What do you mean "same router, different subnet"? Are you using VLANs?
What is your IP config (IP, netmask, gateway, dns) for the Router, PC and nextcloud ?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 12, 2021, 06:01:56 am
Do any of these calls go via JRiver's servers in the US?
No.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 04:02:16 pm
What do you mean "same router, different subnet"? Are you using VLANs?
What is your IP config (IP, netmask, gateway, dns) for the Router, PC and nextcloud ?

Hi Zybex, I've sent you a pic of how my setup.  I forgot to mention that I port forward from the router:
: Port 80 --> Nextcloud Server
: Port 2220 and 21999 --> MC Server
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 04:04:34 pm
Here's an MP4 over HTTPS+DynDNS: Olaf's Frozen Adventure (https://troti.net:52200/MCWS/v1/Share/Get?File=V9Q5pZjenSFuvwxEA6%2FpfCkDTg34%2FPJVVLtxs6zfj5nZfle8DaKoTRsKeXa8fZDt%0AKvezDJL%2FNq8ZmoAx7atGaw%3D%3D)

FYI - Testing of the speed, this morning I'm only getting under 200kbs download speed on this link.

I'll post some video test links shortly (both MC and Nextcloud)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 04:16:54 pm
Here are some links of a 30sec video

MC Links
http://144.137.208.82:52199/MCWS/v1/Share/Get?File=8dlSvKW3vpF1QpmlzE%2BUZ804uzsI1XixknlNTB9VrSCvwmOUzOZx2fBsAfKzeLkX%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=8dlSvKW3vpF1QpmlzE%2BUZ804uzsI1XixknlNTB9VrSCvwmOUzOZx2fBsAfKzeLkX%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

NextCloud Link
https://behome.dyndns.info/index.php/s/LLrKNdnbNf9QJPW
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 04:36:03 pm
There seems to be some upload speed limit on MC's WebServer. I didn't check with Wireshark, but if behaves like it's not scaling the TCP window (so speed doesn't increase).

I used wget to pull all 3 files from your links:
- NextCloud: Peak over 5MB/sec, average 4.65 MB/sec
- MC HTTP: very stable around 195 KB/sec
- MC HTTPS: very stable around 145 KB/sec

When using the browser, the HTTPS link still takes 2 or 3 minutes. With Wget it starts download immediately, but responds with 206 (Partial Content) instead of 200 OK - this may be causing the browser to attempt to buffer the entire file, which might explain the playback delay.

Something's fishy...

EDIT: I've retried now with the browser and the HTTPS link worked, with some buffering delay (but not 3 minutes this time).
This seems to be the same you are getting with my links - it works, but it's slow. The speed limit is apparently on MC's web server.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 04:47:13 pm
Thanks for all your testing. 

- I too only see <200 KB/s when using your SSL link when download the video file
- When playing your SSL Link I get maybe a 20sec delay as the web browser buffers the initial part then I can commence playing.  It then buffers / plays / butters etc

So it looks like there could be a couple of unrelated issues
1) Performance of the MC Server?
2) In my (unique) case I also had a 2nd ISP Provider router in the path (to do 4G Failover if the main FTTP line failed).  I've now removed that and it has (for me) fixed the odd 3min wait time on the SSL line (no idea why it would cause this but It's removed for now) and directly exposed my Main Router to the WAN IP instead of going through the other routers DMZ. 

Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JustinChase on February 12, 2021, 04:48:28 pm
So, I've got a bit further:
- Created a new DynDNS name for my IP Address
- Created a SSL certificate for the DynDNS name using "Certify the Web" GUI / Lets Encrypt
- Loaded the SSL certificate and key into MC

Link1: Here is the non-ssl (std Share link):  Works and the image displays pretty well straight away

http://144.137.208.82:52199/MCWS/v1/Share/Get?File=wUgLY6bHiVXo8ImCBhqPU3VNbble%2FmCo%2BRzTRaMN5XyPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

Link2: Here is the same link but changed to use HTTPS, MC SSL Port, & DynDNS:  It eventually loads...... but it takes ages (eg a few minutes) to show the picture. 
https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=wUgLY6bHiVXo8ImCBhqPU3VNbble%2FmCo%2BRzTRaMN5XyPSTKGLRjyoRxg1F5uMoYC%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

EDIT:
- If anyone can test by clicking on the two links and letting me know if they load about the same? 
- Oddly, If I access this link from outside my LAN (eg using my phone over a 4G etc) then it is quick.  It is only deadly slow if my device is connected to my LAN (wired or wifi).  Something weird ....

both are similarly fast for me.  I use Firefox nightly, with this option checked "Enable HTTPS-Only Mode in all windows"

So, your first link gives me a dialog warning to use the HTTP version, which does work fine.  The second link opens in about a second.



Here are some links of a 30sec video

MC Links
http://144.137.208.82:52199/MCWS/v1/Share/Get?File=8dlSvKW3vpF1QpmlzE%2BUZ804uzsI1XixknlNTB9VrSCvwmOUzOZx2fBsAfKzeLkX%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=8dlSvKW3vpF1QpmlzE%2BUZ804uzsI1XixknlNTB9VrSCvwmOUzOZx2fBsAfKzeLkX%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

NextCloud Link
https://behome.dyndns.info/index.php/s/LLrKNdnbNf9QJPW

the first 2 open quickly, both about a second.  the last link opens a page offering me to download the video, and takes 2-3 seconds to load.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 12, 2021, 04:55:33 pm
Can someone else please share a small video so that we can test performance? Also share your upload connection speed.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 12, 2021, 06:09:42 pm
1) Performance of the MC Server?
Unlikely.  It's just serving a file.  Not a heavy load at all.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: dtc on February 12, 2021, 06:27:24 pm
Here are some links of a 30sec video

MC Links
http://144.137.208.82:52199/MCWS/v1/Share/Get?File=8dlSvKW3vpF1QpmlzE%2BUZ804uzsI1XixknlNTB9VrSCvwmOUzOZx2fBsAfKzeLkX%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=8dlSvKW3vpF1QpmlzE%2BUZ804uzsI1XixknlNTB9VrSCvwmOUzOZx2fBsAfKzeLkX%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

NextCloud Link
https://behome.dyndns.info/index.php/s/LLrKNdnbNf9QJPW

All the links buffer badly. I downloaded the first link and it took 5 minutes to download. Using Chrome with Windows 10 on Comcast.  Speedtest gave 120 Mbps while I was downloading.  Clearly being throttled somewhere.  Amazon Prime plays HD movies fine.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 12, 2021, 10:14:07 pm
All the links buffer badly. I downloaded the first link and it took 5 minutes to download. Using Chrome with Windows 10 on Comcast.  Speedtest gave 120 Mbps while I was downloading.  Clearly being throttled somewhere.  Amazon Prime plays HD movies fine.

That was probably me swapping routers.  I had to set QOS/Shaper on my router for the upload @ 50mbps as I was hitting the FTTP policer (Australian NBN) and it was throttling my upload (down to only 10mbps).  That part should now be "fixed" now from what I can see.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: dtc on February 13, 2021, 06:52:04 am
That was probably me swapping routers.  I had to set QOS/Shaper on my router for the upload @ 50mbps as I was hitting the FTTP policer (Australian NBN) and it was throttling my upload (down to only 10mbps).  That part should now be "fixed" now from what I can see.

The first 2 link still buffer badly. I get 1 or 2 seconds of video with then 15+ seconds of loading.  With the Nextcloud link,  the train eventually  moves, but there is still regular buffering.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 04:29:18 pm
Thanks for the testing.  Looks like I'll need to use lower max bit rates to prevent buffering esp for links that will be used outside of Australia (or on mobile devices).  Here is the same file encoded at 15 and 10Mb/s (instead of 20Mb/s) and in order they are MC using HTTP, MC using SSL, and NextCloud.

Note:  NextCloud seems to be much faster than MC (that is another discussion) so I expect it will work way better at present.

15Mb/s Clip

http://144.137.208.82:52199/MCWS/v1/Share/Get?File=EcnkaFQ6xO77r0CgJXZtHxSlffCMZV0kltdt6rF7LsbtgZuQ4oONqoPhwN1i1NLs%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=EcnkaFQ6xO77r0CgJXZtHxSlffCMZV0kltdt6rF7LsbtgZuQ4oONqoPhwN1i1NLs%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://behome.dyndns.info/index.php/s/3RLd2yAzHDyz8ag

10Mb/s Clip

http://144.137.208.82:52199/MCWS/v1/Share/Get?File=4UOaiDmqW12aqG5ZXlgyFTOHzOkC71bAAshkkV3gERqIiCK9xveHqeYMnQThhLg9%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://mymc.dyndns.info:52200/MCWS/v1/Share/Get?File=4UOaiDmqW12aqG5ZXlgyFTOHzOkC71bAAshkkV3gERqIiCK9xveHqeYMnQThhLg9%0AICd0bok%2BZ7cUrJmzKQcSDg%3D%3D

https://behome.dyndns.info/index.php/s/ALBZAqywKPSixSM
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: dtc on February 13, 2021, 04:39:46 pm
Unfortunately, these are not much better. And, yes, I am in the US.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 04:45:25 pm
One final test (to see how much of the speed limit is my home connection VS the International Link out of Australia) -

https://jmone.org/jmonenextcloud/index.php/s/G8GAprofqfaKwsb

This one is hosted in an Australian data centre. 

Thanks for all the testing!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: dtc on February 13, 2021, 04:55:34 pm
No buffering - the train has left the station!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 05:10:21 pm
Well that is very interesting.  The data centre has much higher bandwidth of course, but my 50Mb/s home connection should be plenty for any of these clips, and they all work fine from Australian based clients.  I guess it could be congestion/throttling on residential connections out of Oz Vs their T1 connection ... or something else (thinking thinking).
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 05:17:23 pm
I just noticed MC serving a file to "red.jriver.com" - Jim was that you? :)
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: dtc on February 13, 2021, 05:49:23 pm
I have Comcast as my Internet provider in the US. My download speed is 120 Mbps, but upload is around 5 Mbps.  That is pretty common for Comcast users. You need a Gigabit download plan to get above 10 Mbps upload.  I am not sure this Share feature will work well at those upload speeds. Comcast is the largest cable provider in the US.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 13, 2021, 06:33:01 pm
I just noticed MC serving a file to "red.jriver.com" - Jim was that you? :)
Yes, but my office has a wi-fi link so it didn't stream without pausing.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 13, 2021, 06:36:37 pm
The nominal speeds aren't reliable.  Try running an Internet speed test (https://support.google.com/websearch/answer/6283840?p=speedtest&visit_id=637488596684658151-2012735645&rd=1) a few times.

I have a pretty slow connection at home because it's enough for me.  But when I run the speed test, I get results that vary a lot.  I think it's the girls at the end of the street watching movies in their rooms.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 07:09:16 pm
I think this is a great feature as Upload Speeds are only going to increase with time and with it the need to host stuff on 3rd party servers will diminish. 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: dtc on February 13, 2021, 09:33:17 pm
The nominal speeds aren't reliable.  Try running an Internet speed test (https://support.google.com/websearch/answer/6283840?p=speedtest&visit_id=637488596684658151-2012735645&rd=1) a few times.

I have a pretty slow connection at home because it's enough for me.  But when I run the speed test, I get results that vary a lot.  I think it's the girls at the end of the street watching movies in their rooms.

The speeds I am discussing are measured speeds.  There are pretty consistent at 120 Mbps down and 5 Mbps up, versus Comcast's stated 100 and 5 Mbps.  That is the nature of Comcast service, partially because their final service to the home is still mostly copper, not fiber.  I really do not see a great jump in upload speed from Comcast in the near future. It is just not a priority for them.  Verizon designed their system to deliver good upload and download speeds, but Comcast is just not there at this time.  And they have little incentive to upgrade that speed with their existing systems, as they concentrate on deploying 5G to the home.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 09:39:31 pm
Mine is very consistently 930 / 47 Mb/s  (when I don't break things  ::) ) .... I'm thinking that I'd be better off with a more symmetrical connection, but for us these plans cost a fair bit more (eg 500/100 or 200/200) 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 13, 2021, 09:54:45 pm
I feel for you guys... I really do.   :)

(https://i.vgy.me/IGMmFl.jpg)

It's actually difficult to find test servers that have adequate bandwidth.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 13, 2021, 10:03:30 pm
No one likes you  :P !!!!

1G symmetrical here in Oz would be the better part of US$1K pm. 
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: JimH on February 13, 2021, 10:30:49 pm
Who cares?
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 14, 2021, 06:39:59 am
Nice photoshop skillz, Wer  ;D

/crying slowly at 25Mbps
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 19, 2021, 02:04:44 am
Did some testing of Share / Unshare function
- Each time you "Share" you get a new link, but all links works
- Unshare / Uncheck the "Shared" Tick box stops the links from working
- Checking the "Shared" / Sharing will turn sharing back on and all the links works

Any thoughts on this behaviour?  I'm in 2 minds as:
- You could end up with multiple links to the same file
- Once shared, the links will always work if sharing is on.  You can't "clear" or forget the links.  This stops you from changing who can see an item.   Eg I might have a video I've sent a Share Link to both my Family and Friends, but at some point I may only my Family to see it.  At present there is no way to do this, as if I generate a new link for the Family, the Old one will still work.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 19, 2021, 02:49:54 am
Yes, that is by design. The unencrypted share link for a given file is always the same, but each time it's generated there's a seed/timestamp added which completely changes the encrypted string, so each time the link looks different but the embedded info is the same. All links work because when decrypted they point to the same FileKey.

To have different links go to different people, and to be able to invalidate each one individually would require all generated links to be tracked in the DB (with a friendly name) so that you could invalidate each one individually. That's doable but not very nice. Another way would be to create a lists of accounts/passwords, and track the list of items shared with each account; remote users would need to login to see the shared file(s).

I think the best way forward would be to add the concept of Shared Playlist. That way, you would:
- create a playlist for each person/family/group you wish to share
- get a Share URL for the playlist itself
- add/remove files from the playlist whenever you wish without having to explicitly share each file

Accessing the Playlist shared URL would display Panel, with just that playlist visible.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 19, 2021, 03:05:09 am
I think the best way forward would be to add the concept of Shared Playlist...

Brilliant!  If only someone had thought of that before... 

It seems to me that if you have a single playlist or smartlist named "Public" it will be shared...
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 19, 2021, 03:23:03 am
Well, if you're pulling credits for discarded ideas you'll need to read the 2 posts above that one, where I also mentioned sharing playlists. A full 12 minutes before you did  ;D

What about sharing playlists instead of files? We could have different playlists for different people, different credentials/key per playlist, etc. Seems more practical than distinct libraries. This could work for any generic playlist, or you could add a special Playlists\Shared section in the tree. Playlist Groups could also be used to share a group of playlists to the same person.
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: wer on February 19, 2021, 03:37:50 am
See, you agree! It's brilliant every time someone says it!  ;D
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: zybex on February 19, 2021, 03:39:14 am
Indeed. Jim also agreed ("we need both"), so hopefully this is just a matter of time. Baby steps!
Title: Re: Idea: Public URL for external streaming from MC (AKA MC YouTube)
Post by: jmone on February 28, 2021, 03:38:09 am
...another great feature is you can use "Notes" to keep track of who you shared an item with.... including a Clickable URL!  Nice.  So If I ever need to reset the links I have a record of what posts or people I need to update.