I have a custom layout and in my VideoHLSView comp...
# ask-ai
h
I have a custom layout and in my VideoHLSView comp, if i add a video to stream for my viewer, i have latence and bug + it's not responsive, any answer ? (React)
t
Thanks for reaching out, Alexandre. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I hear your frustration with latency, bugs, and responsiveness when adding a video in your custom
VideoHLSView
React component. I can help, but I need a bit more detail to diagnose the problem and give targeted steps. Could you share the following? 1. Which LiveKit React SDK version are you using? 2. Is the stream coming from LiveKit as WebRTC or as HLS (an HLS manifest / m3u8)? If HLS, how are you loading and rendering it (native <video> tag, hls.js, video.js, etc.)? 3. A short code snippet (or description) of how you add the video to the viewer in
VideoHLSView
— specifically how you attach the source/track to the element and any relevant state/props. 4. Which browsers and devices show the latency/unresponsiveness? Is it consistent across them? 5. What do you mean by “latence and bug” — is it high end-to-end delay, frame drops, audio/video out of sync, frozen UI, or console errors? If you have console logs or network traces (e.g., HLS segment download times) those help a lot. 6. Approximate network conditions when this happens (good Wi‑Fi, cellular, high packet loss?) and whether you’ve tried testing on a local network. 7. Any CSS/layout code that affects the video element (responsive behavior often comes from CSS or container sizing). Based on common causes, here are a few things you can check meanwhile: • If you’re using HLS in the browser, ensure you use a player like hls.js for browsers that don’t support native HLS. Misconfigured HLS players can cause large latency or playback stalls. • For HLS latency: confirm your HLS stream is configured for low-latency (segment duration, manifest updates). Long segment durations increase latency. • If you’re attaching a LiveKit WebRTC track to a video element: make sure you’re assigning the MediaStream to
video.srcObject
only once and not causing re-mounts that recreate the element frequently. • Avoid frequent React re-renders of the video element; keep the video element stable and update only attributes that must change. Use refs to manage
srcObject
. • Check CSS for fixed heights/widths or layout thrashing; use responsive rules (max-width, aspect-ratio) and avoid heavy layout calculations on render. • Watch the browser console for decode/warning errors and the network tab for stalled segment requests. If you paste the minimal code for
VideoHLSView
and tell me whether the stream is HLS or WebRTC, I’ll give step-by-step fixes and a small example showing the recommended way to attach the stream and keep the component responsive. Sources: Bringing Zoom&amp;#39;s end-to-end optimizations to WebRTC | React Core and Components
h
In my case, i have a responsive view, it's possible to have different flux with breakpoint (3 max) ?
t
Thanks — that’s a great question. Yes, you can absolutely have a responsive view that shows different numbers of video streams (flux) depending on breakpoints (for example: 1 on mobile, 2 on tablet, 3 on desktop). Below I’ll briefly explain the approach, give a small React pattern you can adopt, and point out a few performance tips so the layout stays responsive. Acknowledgement • You asked whether it’s possible to render different numbers of streams up to 3 depending on breakpoints. That is possible and commonly done. How to approach this (high level) • Detect the viewport breakpoint (CSS media queries or JS with
window.matchMedia
/ resize listener) to determine how many streams to show. • Keep the video DOM nodes stable (use refs) and only change which tracks are attached instead of re-creating elements on every render. • Optionally use selective subscription (subscribe only to the tracks you render) to save bandwidth and reduce decoding work. Small React pattern (responsive max streams + stable video refs) 1. Create a hook to return the max number of streams for the current viewport. 2. Render a fixed number of video containers (1–3) and attach the chosen participant track to each using refs. 3. When breakpoint changes, swap attached tracks into the existing video elements rather than remounting. Example outline (adapt to your stack / LiveKit hooks):
Copy code
import React, { useEffect, useRef, useState } from 'react';
import { useRoom } from '@livekit/react-core'; // or your room hook

function useMaxStreams() {
  const [maxStreams, setMaxStreams] = useState(3);
  useEffect(() => {
    const update = () => {
      if (window.innerWidth < 600) setMaxStreams(1);
      else if (window.innerWidth < 1024) setMaxStreams(2);
      else setMaxStreams(3);
    };
    update();
    window.addEventListener('resize', update);
    return () => window.removeEventListener('resize', update);
  }, []);
  return maxStreams;
}

export default function ResponsiveVideoGrid() {
  const { participants } = useRoom(); // array of participants (or tracks) from LiveKit
  const max = useMaxStreams();

  // Prepare an array of refs equal to max possible slots (3)
  const videoRefs = [useRef<HTMLVideoElement | null>(null), useRef<HTMLVideoElement | null>(null), useRef<HTMLVideoElement | null>(null)];

  // Choose which participant tracks to show (e.g., top N)
  const toShow = participants.slice(0, max);

  useEffect(() => {
    // Attach tracks to stable video elements via refs
    toShow.forEach((p, i) => {
      const videoEl = videoRefs[i].current;
      const track = p.getTrackPublication('video')?.track;
      if (videoEl && track && track.mediaTrack) {
        // If track is a MediaStreamTrack (WebRTC), attach via srcObject
        const ms = new MediaStream([track.mediaTrack]);
        if (videoEl.srcObject !== ms) {
          videoEl.srcObject = ms;
          videoEl.play().catch(() => {});
        }
      } else if (videoEl) {
        // clear if no track
        if (videoEl.srcObject) {
          videoEl.srcObject = null;
        }
      }
    });
  // run when participants or max change
  }, [toShow.map(p => p.sid).join(','), max]);

  return (
    <div style={{ display: 'grid', gap: 8, gridTemplateColumns: `repeat(${max}, 1fr)` }}>
      {Array.from({ length: max }).map((_, i) => (
        <video
          key={i}
          ref={videoRefs[i]}
          style={{ width: '100%', height: 'auto', objectFit: 'cover' }}
          muted
          playsInline
          autoPlay
        />
      ))}
    </div>
  );
}
Practical tips to keep it responsive and low-latency • Keep video elements stable: avoid conditional rendering that unmounts/remounts the <video> element frequently. • Use selective subscription: subscribe only to the tracks you actually render to reduce CPU and network load. • For HLS streams: use a proper HLS player (hls.js) and tune segment duration for lower latency. For WebRTC tracks, ensure you’re not repeatedly reassigning
srcObject
. • Avoid heavy layout work on resize; debounce expensive operations. • Use
object-fit: cover
or
contain
plus CSS aspect ratio to maintain a responsive and consistent look. • Test on low-powered devices and adjust default resolution or simulcast layers if needed. If you want, next I can: • Provide a variant that uses conditional CSS grid layouts or CSS-only breakpoints. • Show an example that selectively subscribes/unsubscribes tracks when the breakpoint changes (saves bandwidth). • Review your
VideoHLSView
code snippet and suggest specific fixes to reduce latency and make it responsive. Which would you like next, and can you share a short snippet of your current
VideoHLSView
(how you attach the source/track and any CSS) so I can tailor the fix? Sources: React Core and Components | React Core and Components | Optimizing Video Calls for Low-Powered Devices in LiveKit (JS SDK ≥ v2.14.0... | Optimizing Video Calls for Low-Powered Devices | LiveKit
h
So, it's mandatory to have 3 flux HLS ? It's not possible with juste one flux ?
t
Good question — you do NOT have to use three HLS fluxes. A single HLS stream can work; multiple fluxes (renditions/variants) are optional and are typically used for adaptive bitrate or to show different quality/resolutions at different sizes. A few points to help you choose: • Single HLS stream ◦ Works fine if you only need one quality level. ◦ Simpler to implement and easier to manage. ◦ May cause higher stalls or poor quality on constrained networks or low-power devices because there’s no lower-bitrate fallback. • Multiple HLS renditions (recommended when possible) ◦ Provide adaptive bitrate: the player can switch to a lower bitrate on slow networks or smaller screens to reduce stalls and latency. ◦ Useful for responsive layouts where desktop might use high resolution and mobile should use a lower one. ◦ Adds encoding complexity and more bandwidth on the origin side, but reduces playback issues for viewers. What I recommend next • Tell me what you currently have: ◦ Is your stream a single HLS manifest (.m3u8) with one variant, or does it include multiple variant streams? ◦ Are you using a player like hls.js or the native <video> tag? ◦ What’s your main goal: lowest latency, simplest implementation, or best quality across devices? If your goal is simplicity and you can accept one fixed quality, stick with one HLS flux. If you need robust playback across varying networks and responsive breakpoints, consider providing multiple renditions (or use LiveKit/WebRTC with simulcast if you need very low-latency interactive streams). If you share your current HLS setup or player code, I’ll recommend the exact changes for either single-flux or multi-rendition workflows.
h
I use a signel HLS but i have 3 layouts so it's not responsive when i resize my viewer screen (my presenter screen is on desktop) , i using the native video tag and best quality across devices
t
Thanks — that helps. You can keep a single HLS stream and still support three responsive layouts. You don’t need multiple HLS renditions to make your UI responsive; renditions only help with adaptive bitrate/quality across network/device conditions. Below are practical steps and examples to make a native <video> element responsive across your three layouts while keeping best quality. Key ideas (short) • Make the video element size responsive to its container (not the other way around). • Keep the video element stable in the DOM and only change layout/CSS on resize to avoid reattaching srcObject or reloading the stream. • Use CSS (media queries, grid/flex, aspect-ratio, object-fit) to implement your 3 breakpoint layouts. • If network/device variability becomes an issue later, add renditions for adaptive bitrate; not required for layout. CSS patterns for responsive video • Use a container that defines the layout and aspect ratio. • Use
object-fit
to control cropping vs letterboxing. • Use
aspect-ratio
or padding-top trick to preserve height relative to width. Example CSS (three breakpoints: mobile, tablet, desktop)
Copy code
.video-grid {
  display: grid;
  gap: 8px;
}

/* Mobile: single column (1 flux) */
@media (max-width: 599px) {
  .video-grid {
    grid-template-columns: 1fr;
  }
}

/* Tablet: two columns (2 flux) */
@media (min-width: 600px) and (max-width: 1023px) {
  .video-grid {
    grid-template-columns: repeat(2, 1fr);
  }
}

/* Desktop: three columns (3 flux) */
@media (min-width: 1024px) {
  .video-grid {
    grid-template-columns: repeat(3, 1fr);
  }
}

/* Maintain aspect ratio & responsive video */
.video-item {
  width: 100%;
  aspect-ratio: 16 / 9; /* or 4/3 depending on your source */
  background: black;
  position: relative;
  overflow: hidden;
}

.video-item video {
  width: 100%;
  height: 100%;
  object-fit: cover; /* or contain to avoid cropping */
  display: block;
}
React approach (keep video DOM stable) • Render a fixed set of video slots (max 3) and attach the HLS stream to the relevant video element via a ref. • When layout changes, move which slot is visible or change CSS—do not unmount/remount the video element unnecessarily. Example outline 1. Create three video slot elements (always present). 2. Use CSS grid to show 1/2/3 columns based on viewport. 3. Attach the HLS source to the main video ref (or to whichever slot you choose). If using one HLS stream shown across different slots, attach it to the currently visible element and remove from hidden ones without reloading the source when possible. If you’re using native <video> + HLS: • Native HLS works on Safari. For other browsers use hls.js to load .m3u8 into a video element without reloading the stream on layout changes. • With hls.js, attach the HLS instance to the video element once and only call
hls.attachMedia(video)
again if you truly need to switch elements. Prefer keeping the same video element and reflowing it via CSS. Performance tips • Avoid frequent reassignments of
video.src
or re-initializing hls.js on every resize. • Debounce resize handlers if you need JS to react to layout changes. • Use a lower-resolution CSS
object-fit: cover
approach to keep visuals consistent; add renditions later if you see stalls on weaker networks. • If presenter is desktop and viewers are various devices, test mobile under slower network to ensure single-flux quality is acceptable. Next steps I can help with • Review a small snippet of your current layout/CSS (your
VideoHLSView
or the grid CSS) and suggest precise edits. • Show a short example of using hls.js with a single video element and CSS responsive layout so you don’t reload the stream on breakpoints. • Show how to switch which participant’s stream is shown in a slot without reloading the stream source. Which would you like me to do next? If you paste your current
VideoHLSView
markup/CSS (or tell me if you’re using hls.js or native HLS), I’ll give a concrete patch.
h
const RecordingStage = ({ onMediaReady, backgroundColor, theme = 'LIGHT', }: { onMediaReady: () => void; backgroundColor?: string; theme?: 'LIGHT' | 'DARK'; }) => { const nameBadgeClass = theme === 'DARK' ? 'bg-[#17805C] text-white' : 'bg-decorative-1 text-black'; const room = useRoomContext(); const participants = useParticipants(); const [layout, setLayout] = useState<{ type: string; active: { type: string; url: string; page?: number; paused?: boolean; muted?: boolean; currentTime?: number; } | null; }>({ type: 'grid', active: null, }); useEffect(() => { const sync = (metadata?: string) => { if (!metadata) return; try { const data = JSON.parse(metadata); setLayout({ type: data.layout || 'grid', active: data.activeAsset || null }); } catch (e) { console.error('Metadata error', e); } }; sync(room.metadata); room.on(RoomEvent.RoomMetadataChanged, sync); return () => { room.off(RoomEvent.RoomMetadataChanged, sync); }; }, [room]); const presenters = useMemo( () => participants.filter( (p) => !p.isLocal && (p.permissions?.canPublish || p.getTrackPublications().length > 0), ), [participants], ); const presenterItems = useMemo( () => presenters.map((participant) => { const pub = participant.getTrackPublication(Track.Source.Camera); const camTrack = pub ? { participant, source: Track.Source.Camera, publication: pub } : undefined; const displayName = participant.name || participant.identity || ''; return { participant, camTrack, displayName }; }), [presenters], ); const isPres = layout.type === 'presentation' && layout.active; const hasPresenters = presenterItems.length > 0; const presenterGridCols = presenterItems.length >= 4 ? 'grid-cols-4 items-stretch' : `grid-cols-${presenterItems.length} items-stretch`; const videoRef = useRef<HTMLVideoElement>(null); useEffect(() => { if ( layout.type !== 'presentation' || layout.active?.type !== 'VIDEO' || !layout.active?.url || !videoRef.current ) return; const video = videoRef.current; const handleCanPlay = () => { publishPresentationAudioOnly(video, room).catch(console.error); }; video.addEventListener('loadeddata', handleCanPlay); if (video.readyState >= 2) handleCanPlay(); return () => { video.removeEventListener('loadeddata', handleCanPlay); unpublishPresentationAudio(room).catch(console.error); }; }, [layout.type, layout.active?.type, layout.active?.url, room]); useEffect(() => { if (layout.type !== 'presentation' || layout.active?.type !== 'VIDEO' || !videoRef.current) return; const video = videoRef.current; const { paused, muted, currentTime } = layout.active; if (typeof paused === 'boolean') { if (paused) { video.pause(); } else { video.play().catch(() => {}); } } if (typeof muted === 'boolean') { video.muted = muted; } if (typeof currentTime === 'number' && !Number.isNaN(currentTime)) { const delta = Math.abs(video.currentTime - currentTime); if (delta > 0.5) { video.currentTime = currentTime; } } }, [ layout.type, layout.active?.type, layout.active?.paused, layout.active?.muted, layout.active?.currentTime, ]); return ( <div className="w-full h-full min-h-screen overflow-hidden" style={{ backgroundColor: backgroundColor || '#FFFFFF' }} > {layout.type === 'presentation' && layout.active && layout.active?.type === 'VIDEO' && ( <div className="flex flex-col-reverse w-full h-full gap-2"> <div className="flex flex-row items-center justify-center gap-3"> {presenterItems.length > 0 ? ( presenterItems.map(({ participant, camTrack, displayName }) => ( <div key={participant.identity} className=" w-[350px] h-[250px] relative bg-black rounded overflow-hidden" > {camTrack ? ( <div className="w-full h-full relative"> <VideoTrack trackRef={camTrack} className="w-full h-full object-cover" onPlaying={onMediaReady} /> {displayName && ( <div className={
absolute bottom-0 left-0 z-60 px-4 py-1 text-[20px] md:text-[16px] lg:text-[20px] font-medium ${nameBadgeClass}
} > {displayName} </div> )} </div> ) : ( <div className="w-full h-full flex items-center justify-center text-gray-500 text-sm italic"> Caméra... </div> )} </div> )) ) : ( <div className="w-[350px] h-[250px] relative bg-black rounded overflow-hidden flex items-center justify-center text-gray-500 text-sm italic"> Caméra... </div> )} </div> <video ref={videoRef} src={layout.active.url} muted={layout.active.muted ?? false} className="w-full h-full object-contain aspect-16/7 rounded-lg" /> </div> )} {layout.type === 'presentation' && layout.active && layout.active?.type === 'PDF' && ( <div className="flex flex-col-reverse w-full h-full gap-2"> <div className="flex flex-row items-center justify-center gap-3"> {presenterItems.length > 0 ? ( presenterItems.map(({ participant, camTrack, displayName }) => ( <div key={participant.identity} className="w-[350px] h-[250px] relative bg-black rounded overflow-hidden" > {camTrack ? ( <div className="w-full h-full relative"> <VideoTrack trackRef={camTrack} className="w-full h-full object-cover" onPlaying={onMediaReady} /> {displayName && ( <div className={
absolute bottom-0 left-0 z-60 px-4 py-1 text-[20px] md:text-[16px] lg:text-[20px] font-medium ${nameBadgeClass}
} > {displayName} </div> )} </div> ) : ( <div className="w-full h-full flex items-center justify-center text-gray-500 text-sm italic"> Caméra... </div> )} </div> )) ) : ( <div className="w-[350px] h-[250px] relative bg-black rounded overflow-hidden flex items-center justify-center text-gray-500 text-sm italic"> Caméra... </div> )} </div> <div className="w-full h-full overflow-hidden flex items-center justify-center rounded-lg bg-black/5"> {layout.active?.url ? ( <Document file={layout.active.url} onLoadSuccess={() => onMediaReady()} onLoadError={(e) => console.error(e)} loading={<Skeleton className="w-full h-full rounded-lg" />} error={ <div className="p-4 text-red-600 text-sm">Impossible de charger le PDF.</div> } noData={<div className="p-4 text-gray-500 text-sm">Aucun fichier PDF.</div>} > <Page key={layout.active?.page ?? 1} pageNumber={layout.active?.page ?? 1} renderTextLayer={false} renderAnnotationLayer={false} width={1200} loading={<Skeleton className="w-full h-full rounded-lg" />} /> </Document> ) : ( <div className="text-gray-400 text-xl font-medium"> {layout.active?.type === 'PDF' ? 'Attente de la présentation...' : 'Attente de la vidéo...'} </div> )} </div> </div> )} export default function LiveKitRecordingViewPage() { const [params] = useSearchParams(); const token = params.get('token'); const serverUrl = params.get('url') || import.meta.env.VITE_LIVEKIT_WS_URL; const webinarId = params.get('webinarId'); const sessionIdentifier = params.get('sessionIdentifier'); const dispatch = useAppDispatch(); const currentPublicWebinarSession = useAppSelector( (_state_: RootState) => state.webinar.currentPublicSession, ); const signaled = useRef(false); const intervalRef = useRef<ReturnType<typeof setInterval> | null>(null); const startSignal = useCallback(() => { if (signaled.current) return; signaled.current = true; intervalRef.current = setInterval(() => { (globalThis as any).livekitReady = true; document.body.dataset.livekitReady = 'true'; console.log('START_RECORDING'); }, 500); }, []); useEffect(() => { (globalThis as any).livekitReady = false; const safety = setTimeout(startSignal, 15000); return () => { clearTimeout(safety); if (intervalRef.current) clearInterval(intervalRef.current); }; }, [startSignal]); return ( <LiveKitRoom token={token} serverUrl={serverUrl} connect={true} audio={false} video={false} onDisconnected={() => { console.log('END_RECORDING'); if (intervalRef.current) clearInterval(intervalRef.current); }} options={{ adaptiveStream: false }} > <div id="capture-area" className="w-screen h-screen min-h-screen overflow-hidden" style={{ backgroundColor }} > <RoomAudioRenderer /> <RecordingStage onMediaReady={startSignal} backgroundColor={backgroundColor} theme={theme} /> </div> </LiveKitRoom> ); }
t
Thanks — this is great context. I see your RecordingStage component and that you’re using a single HLS/MP4 URL on a native <video> element. You don’t need multiple HLS fluxes to be responsive. The main issues causing your layout to feel unresponsive are the fixed width/height Tailwind classes (e.g., w-[350px] h-[250px], aspect-16/7) and potentially remounting/reassigning the video element on layout changes. What to change (summary) • Replace fixed pixel sizes with responsive classes (or CSS grid) so the layout adapts to breakpoints. • Keep video DOM nodes stable (don’t unmount/remount). Attach the source once and let CSS handle resizing. • Use
aspect-ratio
+
object-fit
to preserve video proportions across breakpoints. • If you need to show 1/2/3 visible presenter thumbnails based on width, hide extra slots with CSS rather than creating/destroying them in JSX. • If your single HLS stream must move between slots, avoid re-initializing the player on each change; reuse the same video element where possible (or reattach the same MediaStream/HLS instance). Concrete changes you can apply 1) Replace fixed presenter boxes with responsive sizes • Instead of className=" w-[350px] h-[250px]" use classes like: ◦ mobile: w-full aspect-[16/9] ◦ tablet/desktop: fixed column widths via grid Example Tailwind-friendly wrapper:
Copy code
// container for presenters
<div className="grid gap-3
  grid-cols-1
  sm:grid-cols-2
  lg:grid-cols-3
">
  {presenterItems.map(...).map(item => (
    <div key={...} className="w-full aspect-[16/9] bg-black rounded overflow-hidden">
      {/* video or placeholder */}
    </div>
  ))}
</div>
2) Keep the main <video> element stable and responsive • Use
w-full h-full object-contain
or
object-cover
plus
aspect-ratio
on the container. • Don’t change the
src
or replace the
<video>
when switching layouts. If you must show the same source in different DOM positions, move the element in the DOM (CSS grid order) instead of destroying it, or attach the HLS player to a ref once. Example: always render the video element, controlled by CSS grid layout
Copy code
<div className="flex flex-col-reverse w-full h-full gap-2">
  <div className="presenters grid gap-3 grid-cols-1 sm:grid-cols-2 lg:grid-cols-3">
    {/* presenter thumbnails (always render up to 3) */}
  </div>

  <div className="main-video w-full rounded-lg overflow-hidden aspect-[16/9] bg-black">
    <video
      ref={videoRef}
      src={layout.active.url}
      muted={layout.active.muted ?? false}
      className="w-full h-full object-contain"
      playsInline
      autoPlay
    />
  </div>
</div>
3) Avoid reassigning src/rewiring on breakpoints • You already use effects to sync playback state (pause/play/muted/currentTime). That’s good. Make sure those effects don’t cause the <video> element to be recreated. • If you use hls.js on non-Safari: init hls once and call
hls.attachMedia(videoRef.current)
only once. Do not recreate hls on each resize. 4) Show/hide presenter slots with CSS rather than conditional JSX • If you want at most 3 presenters visible but different counts per breakpoint, always render 3 presenter slots and hide extras with responsive utilities: ◦ e.g., a presenter slot gets
hidden sm:block lg:block
based on desired breakpoints, or you can use
col-span
to reflow them. This prevents layout thrash and keeps DOM stable. 5) Performance tips • Debounce any heavy resize logic (you can rely on CSS for layout so JS is minimal). • If you see CPU/decode issues on mobile, consider adding a lower-bitrate rendition later — but that’s for network/CPU mitigation, not layout. • Test on mobile under throttled network and CPU to ensure the single-flux quality is acceptable. Would you like: • A direct patch of your RecordingStage JSX showing the Tailwind replacements and stable video ref usage? (I can modify the snippet you pasted.) • An hls.js example that initializes once and keeps playback alive while CSS handles layout changes? Tell me which you prefer and I’ll give the concrete code patch.