Re: Is this possible - Video/Internet?
- From: "cr88192" <cr88192@xxxxxxxxxxx>
- Date: Sat, 30 May 2009 22:59:22 -0700
"spaceman" <educationandmore@xxxxxxxxx> wrote in message
news:f1205a9c-6134-4864-a9ae-0303bd801c87@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
I know there are software packages like Visual Communicator that will
let you stand in front of a green screen and then show you in front of
a different background for the PC.
I would like to create a website where people (in real time) can do
small music performances or anything else in front of different
backgrounds or even cartoon backgrounds created with something like
Flash.
Is this possible using any language on the Internet (if yes, which
ones)? Or is this something that can only be done on home pc and then
manually mapping it on top of a background. Because I really want
this to be done real time.
real-time video processing IS possible.
infact, greenscreening and mixing in a different background would be
computationally much cheaper than other common video tasks, such as encoding
or decoding the video stream...
as for 'any language on the internet', this depends on what you mean by
this.
if you mean 'JavaScript or Flash in a web-browser', then maybe not
(JavaScript and ActionScript are just not well suited to this type of
coding, nor are they likely provide the right APIs...). as can be readily
seen, web-browsers typically get bogged down even by simple things, such as
having too many animated banner ads...
however, if the user can download an app witten in something like C or C++
(or probably even Java), then there is no real problem (the task then
becomes mostly a matter of coding it up...).
I'd also like the singer or even the viewer be able to move the camera
and see the background from different angles.
Also, what are my options when it comes to these backgrounds, what
types of files do they have to be?
depends...
if you do your own app, then you can support any type of file you might want
to support...
for the lazy, there are libraries around for nearly any common graphics
format, and for the less lazy, it is typically not too difficult to write
ones' own image loaders and savers.
the good old common formats: jpeg, png, bmp, tga, ... are a good start...
(pcx and gif and friends can be skipped, as due to their low quality people
are less likely to use them as backgrounds...).
Can this type of thing be done live? I mean - prepare the background
then maybe you stand in front a green screen then people log into your
channel and watch.
yes, as noted above...
note that you would probably want to do any such processing on the app
user's PC, and then probably send a compressed stream to the server in order
to be streamed back to clients...
but, then, the issue here is the usual problem of streaming:
getting the content streamed to the server, at acceptable quality, and while
minimizing the risk of getting backlogged or stalling (would be BAD, as then
all of the clients would likely see the stall, as well as dealing with any
of their own buffering issues...).
different possibilities exist, one of which could be to have several
redundant streams to the server, and some specially designed protocols to
automatically split and merge content from the streams and attempt to
respond quickly to disruptions (the difficulty will be to respond to a stall
in less than 2x the ping time, which could risk disrupting the stream...).
(actually, part of the problem, I think, is that there seems to be a design
flaw in the Windows TCP/IP stack somewhere, which makes it respond rather
poorly to dropped packets...).
the possible cost though, is that redundant streams (presumably with much of
the same contents sent on each), would reduce available bandwidth (and thus,
total video quality).
before I had considered possibilities, namely applying custom filters and
essentially sending partially non-redundant streams. the server end would
then effectively re-merge the streams. the filter would be designed such
that watchable video could be produced from any of the sub-streams, but
where a stall would effectively manifest itself as a temporary reduction in
video quality (a very simple example would be to split 1 video stream into 4
1/2 resolution streams via an even/odd pixel trick, but more developed
filters also possible such that missing pixels could be more easily
recovered via interpolation, such as using a phase/sub-band filtering as
opposed to a raw spatial coding, using offset pixels and a sort of
linear/differential filtering, ..., but the pixel-splitting approach would
be computationally simplest and cheapest...).
note: that the reason for filtering would be mostly to reduce the risk of
"obvious" artifacts from the dropped streams, such as a sudden
video-offsetting or "blurry band" effect (as, for example, every even or odd
pixel in the image becomes interpolated), ... if every pixel is represented
in every subframe, then the artifacting is reduced, as the quality drop will
effect "every region" of the image, rather than "every other pixel", which
as my reasoning goes, should be much less obvious (simply, the overall
quality drops slightly...).
further refinement is possible if one does their own codec (using plain
video codecs would still produce essentially redundant motion-vectors, ...
but, for higher-quality video, motion vectors are not as likely to be a
serious issue...). (MVs are mostly an issue with lower-bitrate streams,
where they can constitute the majority of the coded stream...).
if done well, the 4 (or more) "redundant" streams would not have too much
more overhead than a single larger stream, but when combined with some
protocol hackiness, could greatly reduce the impact of stalls (but, as a
cost, it would be a little more difficult to try to keep all 4 exactly in
sync, as becomming sufficiently out of sync would render the strategy
ineffective). this would mean sending continuous real-time feedback about
the streams' status, helpfully informing the server to drop frames from 1 or
more streams to get them caught up, or possibly instating a kind of "soft"
buffering (such as intentionally knocking the player back a little, possibly
combined with reducing the bandwidth via sub-frame dropping in order to help
the client catch back up, ...).
how to pull off this idea was something I had idly pondered before
considering the terrible annoyance of stalls when watching internet TV, and
the largely ineffective response of typical players in getting the stream
up-and-running again in a timely manner (AKA: the user has to go and hit
play to make the damn thing reconnect, rather than going through some sort
of 10s timeout or whatever before it notes that the stream is stalled and
not fixing itself, and manually reconnects...).
actually, it would likely be a lot more relevant for video uplinks than
downlinks, as we wouldn't want to force everyone to go download some new and
custom video player, ... similarly, for downlinks it would greatly increase
the per-client overhead on the server.
note: yes, it is possible to do real-time video encoding (I have done
so...), however it is much more difficult to produce good results
(bitrate/quality tradeoff) in real-time, as many steps (such as finding good
motion vectors, ...) are typically time-consuming to do well (very simple
for real-time is to forsake motion vectoring/..., but this produces huge
output, and so would not be ideal for streaming...).
note, this is also for why the whole split/merge thing it make sense to do a
custom codec: a lot more effort could be saved in many of these steps,
allowing faster encoding and transcoding in this case vs more traditional
codecs. the 4 substreams could be handled similar to a single larger stream,
and so some of the processing (such as motion-vectoring) could be shared,
and thus make it faster.
similarly, with a custom codec, on the server side the remerging and
transcoding could be done without needing to redo some of these steps (of
course, this is made harder if the downlink uses a different, presumably
more common, codec).
however, personally, I have never done much with these ideas...
the bigger question, however, is whether any of the above would be
worthwhile...
or such...
Thank you,
Markis
.
- References:
- Is this possible - Video/Internet?
- From: spaceman
- Is this possible - Video/Internet?
- Prev by Date: Re: Is this possible - Video/Internet?
- Next by Date: Re: In the midst of chaos, an adventure can be found Topic Summary: A beginners journey into the realm of overclocking
- Previous by thread: Re: Is this possible - Video/Internet?
- Next by thread: Re: In the midst of chaos, an adventure can be found Topic Summary: A beginners journey into the realm of overclocking
- Index(es):
Relevant Pages
|
