Compare commits

...

16 Commits

Author SHA1 Message Date
87e1d24a86 feat: implement abstract base classes for stats plugins and add example plugin for stats setup and download
All checks were successful
Build and Push Docker Image / build (push) Successful in 12m3s
2025-02-23 22:37:21 +01:00
83480ed3a8 fix: update wait condition in setupStats function to check for visibility of stats content
All checks were successful
Build and Push Docker Image / build (push) Successful in 11m58s
2025-02-23 17:53:03 +01:00
5ec57150b1 docs: update README with additional Docker run command options for shared memory size
All checks were successful
Build and Push Docker Image / build (push) Successful in 12m53s
2025-02-22 23:07:11 +01:00
bd80c1c6cc feat: enhance Docker workflow to support logging in and pushing images to Docker Hub; update README with new run command options
All checks were successful
Build and Push Docker Image / build (push) Successful in 4m11s
2025-02-22 18:34:11 +01:00
9e125bf9f7 feat: include sleep to wait for driver setup logic and remove yaspin
Some checks failed
Build and Push Docker Image / build (push) Has been cancelled
2025-02-22 18:21:56 +01:00
a0cf93a6be feat: enhance setupStats function with retry logic and improve error handling; update entrypoint to run main script as PID 1
All checks were successful
Build and Push Docker Image / build (push) Successful in 12m29s
2025-02-21 20:26:03 +01:00
5c92020169 Add convenience functions, update Docker setup, and integrate Webpack for building WebRTC internals exporter (#3)
All checks were successful
Build and Push Docker Image / build (push) Successful in 12m31s
Reviewed-on: #3
Co-authored-by: Mirko Milovanovic <mir_ko@me.com>
Co-committed-by: Mirko Milovanovic <mir_ko@me.com>
2025-02-21 11:21:14 +00:00
84479c786a feat: update Dockerfile to use Debian slim and implement Python virtual environment for dependency management
All checks were successful
Build and Push Docker Image / build (push) Successful in 4m20s
2025-02-15 16:23:48 +01:00
5636d92474 feat: add health check and heartbeat endpoint to Docker setup
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m26s
2025-02-15 14:28:24 +01:00
0518d1ba48 feat: integrate yaspin for improved logging and error handling in Chrome driver setup
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m5s
2025-02-14 15:32:14 +01:00
f579000a96 docs: simplify Docker commands in README and enhance logging in WebRTC stats exporter
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m16s
2025-02-13 18:31:09 +01:00
752cea15d4 docs: reorganize server setup instructions and create dedicated server README
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m6s
2025-02-12 20:24:29 +01:00
5be06ec11f docs: update README with improved Docker commands and add credits section
All checks were successful
Build and Push Docker Image / build (push) Successful in 2m33s
feat: enhance WebRTC stats collection by logging connection state changes
2025-02-12 19:50:37 +01:00
58369fccaf docs: add reference to MongoDB Docker Compose examples in README
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m5s
2025-02-11 11:17:47 +01:00
4c46af63e3 feat: add MIT License file to the repository
All checks were successful
Build and Push Docker Image / build (push) Successful in 1m8s
2025-02-11 11:12:32 +01:00
755128a5c7 docs: update README to include server setup instructions and clarify port requirements
All checks were successful
Build and Push Docker Image / build (push) Successful in 2m19s
2025-02-10 12:45:25 +01:00
26 changed files with 10168 additions and 224 deletions

View File

@@ -1,4 +1,4 @@
name: Build Docker Image for Pull Request name: Build Docker Images for Pull Request
on: on:
pull_request: pull_request:
@@ -25,3 +25,13 @@ jobs:
platforms: | platforms: |
linux/amd64 linux/amd64
linux/arm64 linux/arm64
- name: Build monolith Docker image
uses: docker/build-push-action@v6.13.0
with:
context: .
tags: ${{ github.repository_owner }}/${{ github.event.repository.name }}:${{ github.event.pull_request.number }}-monolith
file: ./Monolith.dockerfile
platforms: |
linux/amd64
linux/arm64

55
.github/workflows/monolith.yml vendored Normal file
View File

@@ -0,0 +1,55 @@
name: Build and Push Docker Image
on:
push:
branches:
- main
env:
REGISTRY_URL: gitea.kobim.cloud
DOCKERHUB_USERNAME: kobimex
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout repository actions
uses: actions/checkout@v2
- name: Setup Docker Environment
uses: ./.github/actions/setup-docker-environment
- name: Log in to custom Docker registry
uses: docker/login-action@v3.3.0
with:
registry: ${{ env.REGISTRY_URL }}
username: ${{ github.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build and push Docker image to custom registry
uses: docker/build-push-action@v6.13.0
with:
context: .
push: true
tags: ${{ env.REGISTRY_URL }}/${{ github.repository_owner }}/${{ github.event.repository.name }}-monolith:latest
file: ./Monolith.dockerfile
platforms: |
linux/amd64
linux/arm64
- name: Log in to Docker Hub
uses: docker/login-action@v3.3.0
with:
username: ${{ env.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push Docker image to Docker Hub
uses: docker/build-push-action@v6.13.0
with:
context: .
push: true
tags: ${{ env.DOCKERHUB_USERNAME }}/${{ github.event.repository.name }}-monolith:latest
file: ./Monolith.dockerfile
platforms: |
linux/amd64
linux/arm64

7
.gitignore vendored
View File

@@ -297,3 +297,10 @@ __pycache__/
test/ test/
venv/ venv/
.venv/ .venv/
# Node.js
node_modules/
npm-debug.log
yarn-error.log
yarn-debug.log*
background.bundle.js

View File

@@ -1,8 +1,15 @@
FROM python:3.13.1-slim-bookworm FROM debian:bookworm-slim
# Install dependencies # Install Python and curl
RUN apt-get update && apt-get install -y python3 python3-pip python3-venv curl
# Create and activate a virtual environment
RUN python3 -m venv /app/venv
ENV PATH="/app/venv/bin:$PATH"
# Install dependencies with venv
COPY requirements.txt /app/requirements.txt COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt RUN /app/venv/bin/pip install -r /app/requirements.txt
# Copy the application # Copy the application
COPY main.py /app COPY main.py /app
@@ -10,4 +17,7 @@ COPY utils/ /app/utils
WORKDIR /app WORKDIR /app
# Run the application # Run the application
CMD ["python", "main.py"] CMD ["/app/venv/bin/python", "main.py"]
# Healthcheck
HEALTHCHECK --interval=5s --timeout=10s --retries=5 --start-period=5s CMD curl -f http://localhost:9092/heartbeat || exit 1

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Mirko Milovanovic
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

50
Monolith.dockerfile Normal file
View File

@@ -0,0 +1,50 @@
FROM node:22.14.0-bookworm-slim AS build
# Copy the webrtc-internals-exporter files
COPY webrtc-internals-exporter /tmp/webrtc-internals-exporter
WORKDIR /tmp/webrtc-internals-exporter/webpack
# Install dependencies
RUN --mount=type=cache,target=/root/.npm \
npm install
# Build the project
RUN npm run build
FROM selenium/standalone-chromium:129.0
# Install Python-virtualenv
RUN sudo apt-get update && sudo sudo apt-get install -y python3-venv
WORKDIR /tmp
# Install Telegraf
RUN wget -q https://repos.influxdata.com/influxdata-archive_compat.key && \
echo '393e8779c89ac8d958f81f942f9ad7fb82a25e133faddaf92e15b16e6ac9ce4c influxdata-archive_compat.key' | sha256sum -c && \
cat influxdata-archive_compat.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg > /dev/null && \
echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list && \
sudo apt-get update && sudo apt-get install -y telegraf
# Create and activate a virtual environment
RUN python3 -m venv ./venv
ENV PATH="/tmp/venv/bin:$PATH"
# Install dependencies with venv
COPY requirements.txt ./requirements.txt
RUN ./venv/bin/pip install -r ./requirements.txt
# Copy files
COPY main.py .
COPY utils/ ./utils
COPY telegraf.conf ./telegraf.conf
COPY webrtc-internals-exporter /tmp/webrtc-internals-exporter
COPY --from=build /tmp/webrtc-internals-exporter/background.bundle.js /tmp/webrtc-internals-exporter/background.bundle.js
COPY --chown="${SEL_UID}:${SEL_GID}" monolith-entrypoint.sh /opt/bin/collector.sh
# Run the entrypoint
RUN chmod +x /opt/bin/collector.sh
ENTRYPOINT ["/opt/bin/collector.sh"]
# Healthcheck
HEALTHCHECK --interval=5s --timeout=10s --retries=5 --start-period=5s CMD curl -f http://localhost:9092/heartbeat || exit 1

View File

@@ -10,19 +10,12 @@ peertube-collector is a project designed to collect and analyze WebRTC statistic
- Docker Engine Community version is required. To install Docker CE, follow the official [install instructions](https://docs.docker.com/engine/install/). - Docker Engine Community version is required. To install Docker CE, follow the official [install instructions](https://docs.docker.com/engine/install/).
### Ports: ### Ports:
#### Localhost (REQUIRED):
- 4444 (Selenium)
Ports can be opened in the host machine's firewall with: #### External (OPTIONAL PROBABLY NOT NEEDED!!!):
```sh These ports are actively used by selenium and the collector services.
ufw allow from 172.30.0.0/16 to any port 4444
```
#### External (OPTIONAL):
These ports are actively used by selenium and the collector services. By defaut they should not be blocked by the firewall, but if so, they can be opened in the host machine's firewall.
- 50000:60000/udp (WebRTC) - 50000:60000/udp (WebRTC)
- Needed for WebRTC NAT traversal, otherwise the browser will not connect to any peer. - WebRTC NAT traversal requires a range of ports to be open.
The range needs to be fairly large since the port is chosen randomly by the STUN server. The range needs to be fairly large since the port is chosen randomly by the STUN server.
- 27107/tcp (MongoDB) - 27107/tcp (MongoDB)
@@ -31,36 +24,67 @@ Ports can be opened in the host machine's firewall with:
ufw allow 50000:60000/udp ufw allow 50000:60000/udp
ufw allow 27107/tcp ufw allow 27107/tcp
``` ```
## Setup with Docker Compose
---
## Setup
1. Clone the repository: 1. Clone the repository:
```sh ```sh
git clone <repository-url> git clone <repository-url>
cd peertube-collector cd peertube-collector
``` ```
2. Create and configure the environment file based on the `.env.example` file: 2. Create and configure the environment file based on the `.env.example` file:
```sh ```sh
cp .env.example .env cp .env.example .env
``` ```
3. Ajust the firewall settings to allow the necessary ports if needed 3. Ajust the firewall settings to allow the necessary ports if needed
4. Start the Docker containers: 4. Start the Docker containers:
```sh ```sh
docker compose up docker compose up --abort-on-container-failure
```
or in detached mode:
```sh
docker compose up -d
``` ```
The collector will start gathering WebRTC stats from the Selenium container and sending them to the Telegraf service.
To stop the Docker containers run: `docker compose down -v` To stop the Docker containers run: `docker compose down -v`
The collector will start gathering WebRTC stats from the Selenium container and sending them to the Telegraf service. ### Setup with Monolithic image:
1. Clone the repository:
```sh
git clone <repository-url>
cd peertube-collector
```
2. Create and configure the environment file based on the `.env.example` file:
```sh
cp .env.example .env
```
3. Ajust the firewall settings to allow the necessary ports if needed
4. Start the Docker container:
```sh
docker run --rm -p 7900:7900 --env-file .env --name peertube-collector --pull always --shm-size="2g" gitea.kobim.cloud/kobim/peertube-collector-monolith:latest
```
or
```sh
docker run --rm -p 7900:7900 --env-file .env --name peertube-collector --pull always --shm-size="2g" kobimex/peertube-collector-monolith:latest
```
### Environment Variables
| Environment Variable | Service | Default Value | Description |
| ------------------------------- | -------------------- | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `TELEGRAF_HOSTNAME` | telegraf | None, **must** be set | Hostname used to identify the host/user between sessions |
| `TELEGRAF_MONGODB_DSN` | telegraf | `mongodb://stats_user...` | DSN for the MongoDB service |
| `TELEGRAF_MONGODB_DATABASE` | telegraf | `statistics` | Database name for the MongoDB service |
| `VIDEO_URL` | collector | `https://tube.kobim.cloud/...` | URL for the video to be analyzed |
| `HUB_URL` | collector | None | URL for the Selenium Hub. If not set, the local Chrome driver will be used |
| `SOCKET_URL` | collector | `localhost` | Socket URL for Telegraf service |
| `SOCKET_PORT` | collector & telegraf | `8094` | Socket port for Telegraf service |
| `WEBRTC_INTERNALS_PATH` | collector | None | **Absolute** path for WebRTC internals exporter extension. When **not** set the extension path is construced relative to the current main script location. |
| `WEBRTC_INTERNALS_EXPORTER_URL` | WebRTC extension | `http://localhost:9092` | Server URL for the WebRTC internals exporter extension |
Variables can be set in the `.env` file.
An example configuration is provided in the `.env.example` file.
### Monitoring ### Monitoring
A noVNC server is available at [http://localhost:7900](http://localhost:7900/?autoconnect=1&resize=scale&password=secret) to monitor the Selenium container. The password is `secret`. A noVNC server is available at [http://localhost:7900](http://localhost:7900/?autoconnect=1&resize=scale&password=secret) to monitor the Selenium container. The password is `secret`.
@@ -78,6 +102,10 @@ The `docker-compose.yml` file defines the following services:
The `Dockerfile` sets up the Python environment and installs the necessary dependencies to run the `main.py` script. The `Dockerfile` sets up the Python environment and installs the necessary dependencies to run the `main.py` script.
### Monolithic Dockerfile
`Monolith.dockerfile` is a single Dockerfile that combines the Selenium, Telegraf, and Collector services into a single container. This is useful for deployment in a single container environment.
### Main Python Script ### Main Python Script
The `main.py` script sets up the Selenium WebDriver, collects WebRTC stats, and sends them to the Telegraf service. The `main.py` script sets up the Selenium WebDriver, collects WebRTC stats, and sends them to the Telegraf service.
@@ -85,17 +113,10 @@ The `main.py` script sets up the Selenium WebDriver, collects WebRTC stats, and
### WebRTC Internals Exporter ### WebRTC Internals Exporter
The `webrtc-internals-exporter` directory contains a Chromium extension that collects WebRTC stats from the browser. The `webrtc-internals-exporter` directory contains a Chromium extension that collects WebRTC stats from the browser.
It uses Webpack to replace the server collector endpoint with an environment variable.
## Working Project Structure # Credits
``` - [WebRTC Internals Exporter](https://github.com/vpalmisano/webrtc-internals-exporter)
peertube-collector/ - [WebRTC debugging with Prometheus/Grafana](https://medium.com/@vpalmisano/webrtc-debugging-with-prometheus-grafana-254b6ac71063)
├── requirements.txt - [MongoDB Docker Compose examples](https://github.com/TGITS/docker-compose-examples/tree/main/mongodb-docker-compose-examples)
├── telegraf.conf
├── docker-compose.yml
├── Dockerfile
├── main.py
├── .env
└── utils/
└── webrtc-internals-exporter/
```

View File

@@ -3,18 +3,19 @@ services:
container_name: selenium-standalone-chromium container_name: selenium-standalone-chromium
image: selenium/standalone-chromium:129.0 image: selenium/standalone-chromium:129.0
volumes: volumes:
- ./webrtc-internals-exporter:/tmp/webrtc-internals-exporter:ro - build-extension:/tmp/webrtc-internals-exporter
shm_size: "2g" shm_size: "2g"
attach: false attach: false
depends_on:
telegraf:
condition: service_healthy
healthcheck: healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4444/wd/hub/status"] test: ["CMD", "curl", "-f", "http://localhost:4444/wd/hub/status"]
interval: 5s interval: 5s
timeout: 10s timeout: 10s
retries: 5 retries: 5
network_mode: host pull_policy: always
ports:
- "7900:7900"
networks:
- backend
telegraf: telegraf:
container_name: telegraf container_name: telegraf
@@ -25,11 +26,32 @@ services:
- DATABASE=${TELEGRAF_MONGODB_DATABASE:?"Database name is required"} - DATABASE=${TELEGRAF_MONGODB_DATABASE:?"Database name is required"}
- DSN=${TELEGRAF_MONGODB_DSN:?"DSN is required"} - DSN=${TELEGRAF_MONGODB_DSN:?"DSN is required"}
- HOSTNAME=${TELEGRAF_HOSTNAME:?"Hostname is required"} - HOSTNAME=${TELEGRAF_HOSTNAME:?"Hostname is required"}
- SOCKET_PORT=${SOCKET_PORT:?"Socket port is required"}
healthcheck: healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080"] test: ["CMD", "curl", "-f", "http://localhost:8080"]
interval: 5s interval: 5s
timeout: 10s timeout: 10s
retries: 5 retries: 5
pull_policy: always
networks:
- backend
build-extension:
container_name: build-extension
image: node:22.14.0-bookworm-slim
volumes:
- ./webrtc-internals-exporter:/tmp/webrtc-internals-exporter:ro
- build-extension:/tmp/webrtc-internals-exporter-build
working_dir: /tmp/webrtc-internals-exporter-build/webpack
command:
- /bin/bash
- -c
- |
cp -r /tmp/webrtc-internals-exporter/* /tmp/webrtc-internals-exporter-build
npm install
npm run build
environment:
- WEBRTC_INTERNALS_EXPORTER_URL=http://collector
pull_policy: always
networks: networks:
- backend - backend
@@ -44,12 +66,14 @@ services:
condition: service_healthy condition: service_healthy
telegraf: telegraf:
condition: service_healthy condition: service_healthy
build-extension:
condition: service_completed_successfully
environment: environment:
- VIDEO_URL=${VIDEO_URL:?"Video URL is required"} - VIDEO_URL=${VIDEO_URL:?"Video URL is required"}
ports: - SOCKET_URL=telegraf
- "9092:9092" - HUB_URL=http://selenium:4444
extra_hosts: - WEBRTC_INTERNALS_PATH=/tmp/webrtc-internals-exporter
- "host.docker.internal:host-gateway" pull_policy: always
networks: networks:
- backend - backend
@@ -57,4 +81,7 @@ networks:
backend: backend:
ipam: ipam:
config: config:
- subnet: 172.30.0.0/16 - subnet: 172.100.0.0/16
volumes:
build-extension:

389
main.py
View File

@@ -1,13 +1,14 @@
import signal import signal
import json
import time import time
import socket
import logging import logging
import os import os
import argparse
from time import sleep
from functools import partial from functools import partial
from http.server import HTTPServer from http.server import HTTPServer
from utils.PostHandler import Handler from utils.PostHandler import Handler
from utils.ColoredFormatter import ColoredFormatter from utils.ColoredFormatter import ColoredFormatter
from utils.Convenience import *
from bs4 import BeautifulSoup as bs from bs4 import BeautifulSoup as bs
from selenium import webdriver from selenium import webdriver
from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.options import Options
@@ -16,11 +17,20 @@ from selenium.webdriver import ActionChains
from selenium.webdriver.support.wait import WebDriverWait from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec from selenium.webdriver.support import expected_conditions as ec
# Plugin system imports
import importlib
import importlib.util
import inspect
import glob
import sys # Import the sys module
from utils.plugins_base import StatsSetupPlugin, StatsDownloadPlugin
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
args = None
def setupLogger(): def setupLogger():
logging_format = "[%(asctime)s] (%(levelname)s) %(module)s - %(funcName)s: %(message)s" logging_format = "[%(asctime)s] (%(levelname)s) %(module)s - %(funcName)s: %(message)s"
logging.basicConfig(level=logging.INFO, format=logging_format) logging.basicConfig(level=firstValid(args.log_level, os.getenv('LOG_LEVEL'), default='INFO'), format=logging_format) # type: ignore
(logger := logging.getLogger(__name__)).setLevel(logging.INFO) (logger := logging.getLogger(__name__)).setLevel(logging.INFO)
logger.propagate = False logger.propagate = False
(logger_handler := logging.StreamHandler()).setFormatter( (logger_handler := logging.StreamHandler()).setFormatter(
@@ -28,172 +38,283 @@ def setupLogger():
) )
logger.addHandler(logger_handler) logger.addHandler(logger_handler)
def setupArgParser():
parser = argparse.ArgumentParser(description='Collector for PeerTube stats.')
parser.add_argument('-u', '--url', type=str, help='URL of the video to collect stats for.')
parser.add_argument('--socket-url', type=str, help='URL of the socket to send the stats to. Default: localhost')
parser.add_argument('--socket-port', type=int, help='Port of the socket to send the stats to. Default: 8094')
parser.add_argument('--hub-url', type=str, help='URL of the Selenium hub to connect to. If not provided, local Chrome driver will be used.')
parser.add_argument('--webrtc-internals-path', type=str, help='Path to the WebRTC internals extension.')
parser.add_argument('--log-level', type=str, help='Log level to use. Default: INFO')
parser.add_argument('--plugin-dir', type=str, help='Path to the plugin directory.')
return parser
def interrupt_handler(signum, driver: webdriver.Remote): def interrupt_handler(signum, driver: webdriver.Remote):
logger.info(f'Handling signal {signum} ({signal.Signals(signum).name}).') logger.info(f'Handling signal {signum} ({signal.Signals(signum).name}).')
driver.quit() driver.quit()
raise SystemExit raise SystemExit
def setupChromeDriver(): def setupChromeDriver(command_executor: str | None, webrtc_internals_path: str) -> webdriver.Remote | webdriver.Chrome:
logger.log(logging.INFO, 'Setting up Chrome driver.') logger.log(logging.INFO, 'Setting up Chrome driver.')
chrome_options = Options() chrome_options = Options()
#chrome_options.add_argument("--headless") #chrome_options.add_argument("--headless")
chrome_options.add_argument("--no-sandbox") chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--mute-audio") chrome_options.add_argument("--mute-audio")
chrome_options.add_argument("--window-size=1280,720") chrome_options.add_argument("--window-size=1280,720")
#chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--no-default-browser-check") chrome_options.add_argument("--no-default-browser-check")
chrome_options.add_argument("--disable-features=WebRtcHideLocalIpsWithMdns") chrome_options.add_argument("--disable-features=WebRtcHideLocalIpsWithMdns")
#chrome_options.add_argument(f"--load-extension={os.path.abspath(os.path.join(os.path.dirname(__file__), 'webrtc-internals-exporter'))}") chrome_options.add_argument(f"--load-extension={webrtc_internals_path}")
chrome_options.add_argument("--load-extension=/tmp/webrtc-internals-exporter")
chrome_options.add_experimental_option('prefs', {'intl.accept_languages': 'en,en_US'}) chrome_options.add_experimental_option('prefs', {'intl.accept_languages': 'en,en_US'})
#driver = webdriver.Chrome(options=chrome_options) if command_executor is not None:
driver = webdriver.Remote(command_executor='http://host.docker.internal:4444', options=chrome_options) driver = webdriver.Remote(command_executor=command_executor, options=chrome_options)
logger.warning(f'Using Selenium hub at {command_executor}.')
else:
driver = webdriver.Chrome(options=chrome_options)
logger.warning('No Selenium hub URL provided, using local Chrome driver.')
logger.log(logging.INFO, 'Chrome driver setup complete.') logger.log(logging.INFO, 'Chrome driver setup complete.')
return driver return driver
def saveStats(stats: list):
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
logger.log(logging.DEBUG, f'Saving stats: {json.dumps(stats, indent=4)}')
sock.sendto(json.dumps(stats).encode(), ('telegraf', 8094))
sock.close()
logger.log(logging.DEBUG, 'Sent stats to socket.')
except socket.error as e:
logger.error(f'Got socket error: {e}')
def downloadStats(driver: webdriver.Chrome, peersDict: dict):
html = driver.find_element(By.CLASS_NAME ,'vjs-stats-list').get_attribute('innerHTML')
if html is not None:
htmlBS = bs(html, 'html.parser')
else:
raise ValueError("html is None")
stats = htmlBS.find_all('div', attrs={'style': 'display: block;'})
playerStats = {
stat.div.text: stat.span.text.replace('\u21d3', 'down').replace('down/', 'down /').replace('\u21d1 ', 'up').replace('\u21d1', 'up').replace('\u00b7', '-').strip()
for stat in stats
}
keys = list(playerStats.keys())
for stat in keys:
if 'Viewport / Frames' == stat:
viewport, frames = playerStats[stat].split(' / ')
width, height = viewport.split('x')
height, devicePixelRatio = height.split('*')
dropped, total = frames.split(' of ')[0].split()[0], frames.split(' of ')[1].split()[0]
playerStats[stat] = {'Width': int(width), 'Height': int(height), 'Pixel ratio': float(devicePixelRatio), 'Frames': {'Dropped': int(dropped), 'Total': int(total)}}
if 'Codecs' == stat:
video, audio = playerStats[stat].split(' / ')
playerStats[stat] = {'Video': video, 'Audio': audio}
if 'Volume' == stat:
if ' (' in playerStats[stat]:
volume, muted = playerStats[stat].split(' (')
playerStats[stat] = {'Volume': int(volume), 'Muted': 'muted' in muted}
else:
playerStats[stat] = {'Volume': int(playerStats[stat]), 'Muted': False}
if 'Connection Speed' == stat:
speed, unit = playerStats[stat].split()
speedBytes = float(speed) * (1024 ** {'B/s': 0, 'KB/s': 1, 'MB/s': 2, 'GB/s': 3}[unit])
playerStats[stat] = {'Speed': speedBytes, 'Granularity': 's'}
if 'Network Activity' == stat:
downString, upString = playerStats[stat].split(' / ')
down, downUnit = downString.replace('down', '').strip().split()
up, upUnit = upString.replace('up', '').strip().split()
downBytes = convert_to_bytes(down, downUnit)
upBytes = convert_to_bytes(up, upUnit)
playerStats[stat] = {'Down': downBytes, 'Up': upBytes}
if 'Total Transfered' == stat:
downString, upString = playerStats[stat].split(' / ')
down, downUnit = downString.replace('down', '').strip().split()
up, upUnit = upString.replace('up', '').strip().split()
downBytes = convert_to_bytes(down, downUnit)
upBytes = convert_to_bytes(up, upUnit)
playerStats[stat] = {'Down': downBytes, 'Up': upBytes}
if 'Download Breakdown' == stat:
server, peer = playerStats[stat].split(' - ')
server, serverUnit = server.replace('from servers', '').strip().split()
peer, peerUnit = peer.replace('from peers', '').strip().split()
serverBytes = convert_to_bytes(server, serverUnit)
peerBytes = convert_to_bytes(peer, peerUnit)
playerStats[stat] = {'Server': serverBytes, 'Peers': peerBytes}
if 'Buffer State' == stat:
del(playerStats[stat])
if 'Live Latency' == stat:
latency, edge = playerStats[stat].split(' (from edge: ')
latency = sum(int(x) * 60 ** i for i, x in enumerate(reversed([part for part in latency.replace('s', '').split('m') if part])))
edge = sum(int(x) * 60 ** i for i, x in enumerate(reversed([part for part in edge.replace('s', '').replace(')', '').split('m') if part])))
playerStats[stat] = {'Latency': latency, 'Edge': edge}
stats = {
'player': playerStats,
'peers': peersDict,
'url': driver.current_url,
'timestamp': int(time.time() * 1000),
'session': driver.session_id
}
saveStats([stats])
def convert_to_bytes(down, downUnit): def convert_to_bytes(down, downUnit):
return float(down) * (1024 ** {'B': 0, 'KB': 1, 'MB': 2, 'GB': 3}[downUnit]) return float(down) * (1024 ** {'B': 0, 'KB': 1, 'MB': 2, 'GB': 3}[downUnit])
def setupStats(driver: webdriver.Remote, url: str): # Default Plugin Implementations
logger.log(logging.INFO, 'Setting up stats.') class DefaultStatsSetupPlugin(StatsSetupPlugin):
actions = ActionChains(driver) def setup_stats(self, driver: webdriver.Remote, url: str, retries: int = 5) -> webdriver.Remote:
wait = WebDriverWait(driver, 30, poll_frequency=0.2) logger.log(logging.INFO, 'Setting up stats.')
actions = ActionChains(driver)
wait = WebDriverWait(driver, 30, poll_frequency=0.2)
driver.get(url) sleep(2)
wait.until(ec.presence_of_element_located((By.CLASS_NAME, 'vjs-big-play-button'))) for attempt in range(retries):
actions.click(driver.find_element(By.CLASS_NAME ,'video-js')).perform() driver.get(url)
wait.until(ec.visibility_of_element_located((By.CLASS_NAME, 'vjs-control-bar'))) try:
actions.context_click(driver.find_element(By.CLASS_NAME ,'video-js')).perform() wait.until(ec.presence_of_element_located((By.CLASS_NAME, 'vjs-big-play-button')))
statsForNerds = driver.find_elements(By.CLASS_NAME ,'vjs-menu-item') break
actions.click(statsForNerds[-1]).perform() except Exception:
wait.until(ec.text_to_be_present_in_element((By.CLASS_NAME, 'vjs-stats-list'), 'Player')) logger.error(f'Timeout while waiting for the big play button to be present. Attempt {attempt + 1} of {retries}')
actions.move_to_element(driver.find_element(By.CLASS_NAME ,'vjs-control-bar')).perform() if attempt == retries - 1:
logger.log(logging.INFO, 'Stats setup complete.') logger.error('Timeout limit reached. Exiting.')
driver.quit()
raise SystemExit(1)
return driver actions.click(driver.find_element(By.CLASS_NAME ,'video-js')).perform()
wait.until(ec.visibility_of_element_located((By.CLASS_NAME, 'vjs-control-bar')))
actions.context_click(driver.find_element(By.CLASS_NAME ,'video-js')).perform()
statsForNerds = driver.find_elements(By.CLASS_NAME ,'vjs-menu-item')
actions.click(statsForNerds[-1]).perform()
wait.until(ec.presence_of_element_located((By.CSS_SELECTOR, 'div.vjs-stats-content[style="display: block;"]')))
actions.move_to_element(driver.find_element(By.CLASS_NAME ,'vjs-control-bar')).perform()
logger.log(logging.INFO, 'Stats setup complete.')
return driver
class DefaultStatsDownloadPlugin(StatsDownloadPlugin):
def download_stats(self, driver: webdriver.Remote, peersDict: dict, socket_url: str, socket_port: int):
html = driver.find_element(By.CLASS_NAME ,'vjs-stats-list').get_attribute('innerHTML')
if html is not None:
htmlBS = bs(html, 'html.parser')
else:
raise ValueError("html is None")
stats = htmlBS.find_all('div', attrs={'style': 'display: block;'})
playerStats = {
stat.div.text: stat.span.text.replace('\u21d3', 'down').replace('down/', 'down /').replace('\u21d1 ', 'up').replace('\u21d1', 'up').replace('\u00b7', '-').strip() # type: ignore
for stat in stats
}
keys = list(playerStats.keys())
for stat in keys:
if 'Viewport / Frames' == stat:
viewport, frames = playerStats[stat].split(' / ')
width, height = viewport.split('x')
height, devicePixelRatio = height.split('*')
dropped, total = frames.split(' of ')[0].split()[0], frames.split(' of ')[1].split()[0]
playerStats[stat] = {'Width': int(width), 'Height': int(height), 'Pixel ratio': float(devicePixelRatio), 'Frames': {'Dropped': int(dropped), 'Total': int(total)}}
if 'Codecs' == stat:
video, audio = playerStats[stat].split(' / ')
playerStats[stat] = {'Video': video, 'Audio': audio}
if 'Volume' == stat:
if ' (' in playerStats[stat]:
volume, muted = playerStats[stat].split(' (')
playerStats[stat] = {'Volume': int(volume), 'Muted': 'muted' in muted}
else:
playerStats[stat] = {'Volume': int(playerStats[stat]), 'Muted': False}
if 'Connection Speed' == stat:
speed, unit = playerStats[stat].split()
speedBytes = float(speed) * (1024 ** {'B/s': 0, 'KB/s': 1, 'MB/s': 2, 'GB/s': 3}[unit])
playerStats[stat] = {'Speed': speedBytes, 'Granularity': 's'}
if 'Network Activity' == stat:
downString, upString = playerStats[stat].split(' / ')
down, downUnit = downString.replace('down', '').strip().split()
up, upUnit = upString.replace('up', '').strip().split()
downBytes = convert_to_bytes(down, downUnit)
upBytes = convert_to_bytes(up, upUnit)
playerStats[stat] = {'Down': downBytes, 'Up': upBytes}
if 'Total Transfered' == stat:
downString, upString = playerStats[stat].split(' / ')
down, downUnit = downString.replace('down', '').strip().split()
up, upUnit = upString.replace('up', '').strip().split()
downBytes = convert_to_bytes(down, downUnit)
upBytes = convert_to_bytes(up, upUnit)
playerStats[stat] = {'Down': downBytes, 'Up': upBytes}
if 'Download Breakdown' == stat:
server, peer = playerStats[stat].split(' - ')
server, serverUnit = server.replace('from servers', '').strip().split()
peer, peerUnit = peer.replace('from peers', '').strip().split()
serverBytes = convert_to_bytes(server, serverUnit)
peerBytes = convert_to_bytes(peer, peerUnit)
playerStats[stat] = {'Server': serverBytes, 'Peers': peerBytes}
if 'Buffer State' == stat:
del(playerStats[stat])
if 'Live Latency' == stat:
latency, edge = playerStats[stat].split(' (from edge: ')
latency = sum(int(x) * 60 ** i for i, x in enumerate(reversed([part for part in latency.replace('s', '').split('m') if part])))
edge = sum(int(x) * 60 ** i for i, x in enumerate(reversed([part for part in edge.replace('s', '').replace(')', '').split('m') if part])))
playerStats[stat] = {'Latency': latency, 'Edge': edge}
stats = {
'player': playerStats,
'peers': peersDict,
'url': driver.current_url,
'timestamp': int(time.time() * 1000),
'session': driver.session_id
}
super().saveStats([stats], socket_url, socket_port)
# Plugin loading mechanism
def load_plugins(plugin_dir: str) -> tuple[StatsSetupPlugin | None, StatsDownloadPlugin | None]:
"""
Loads plugins from the specified directory.
Args:
plugin_dir: The directory to search for plugins.
Returns:
A tuple containing the loaded StatsSetupPlugin and StatsDownloadPlugin, or (None, None) if no plugins were found.
"""
logger.info(f"Loading plugins from {plugin_dir}")
setup_plugin = None
download_plugin = None
plugin_files = glob.glob(os.path.join(plugin_dir, "*.py"))
# Log the contents of the plugin directory
logger.debug(f"Plugin directory contents: {os.listdir(plugin_dir)}")
for plugin_file in plugin_files:
module_name = os.path.basename(plugin_file)[:-3] # Remove .py extension
logger.debug(f"Loading plugin file {plugin_file}")
try:
spec = importlib.util.spec_from_file_location(module_name, plugin_file)
logger.debug(f"Spec: {spec}")
if spec is None:
logger.warning(f"Can't load plugin file {plugin_file}")
continue
module = importlib.util.module_from_spec(spec)
logger.debug(f"Module: {module}")
if spec.loader is not None:
spec.loader.exec_module(module)
else:
logger.warning(f"Can't load module {module_name} from {plugin_file}")
for name, obj in inspect.getmembers(module):
logger.debug(f"Found member: {name} in module {module_name}")
if inspect.isclass(obj):
if issubclass(obj, StatsSetupPlugin) and obj is not StatsSetupPlugin:
logger.info(f"Found StatsSetupPlugin: {obj.__name__}")
setup_plugin = obj()
logger.debug(f"Loaded StatsSetupPlugin: {obj.__name__} from {plugin_file}")
elif issubclass(obj, StatsDownloadPlugin) and obj is not StatsDownloadPlugin:
logger.info(f"Found StatsDownloadPlugin: {obj.__name__}")
download_plugin = obj()
logger.debug(f"Loaded StatsDownloadPlugin: {obj.__name__} from {plugin_file}")
else:
logger.debug(f"Class {obj.__name__} is not a subclass of StatsSetupPlugin or StatsDownloadPlugin")
else:
logger.debug(f"{name} is not a class")
except Exception as e:
logger.warning(f"Error loading plugin {plugin_file}: {e}")
return setup_plugin, download_plugin
if __name__ == '__main__': if __name__ == '__main__':
args = setupArgParser().parse_args()
setupLogger() setupLogger()
driver = setupChromeDriver() # Load plugins
plugin_dir = firstValid(args.plugin_dir, os.getenv('PLUGIN_DIR'), default=None)
if plugin_dir is None:
logger.info("No plugin directory provided. Using default plugins.")
setup_plugin = None
download_plugin = None
else:
setup_plugin, download_plugin = load_plugins(plugin_dir)
# Use default plugins if none are loaded
if setup_plugin is None:
setup_plugin = DefaultStatsSetupPlugin()
logger.info("Using default StatsSetupPlugin.")
if download_plugin is None:
download_plugin = DefaultStatsDownloadPlugin()
logger.info("Using default StatsDownloadPlugin.")
command_executor = firstValid(args.hub_url, os.getenv('HUB_URL'), default=None)
webrtc_internals_path = firstValid(
args.webrtc_internals_path,
os.getenv('WEBRTC_INTERNALS_PATH'),
default=os.path.abspath(os.path.join(os.path.dirname(__file__), 'webrtc-internals-exporter'))
)
driver = setupChromeDriver(command_executor, webrtc_internals_path)
signal.signal(signal.SIGINT, lambda signum, frame: interrupt_handler(signum, driver)) signal.signal(signal.SIGINT, lambda signum, frame: interrupt_handler(signum, driver))
url = os.getenv('VIDEO_URL') url = firstValid(args.url, os.getenv('VIDEO_URL'), default=None)
if url is None: if url is None:
logger.error('VIDEO_URL environment variable is not set.') logger.error('VIDEO_URL environment variable or --url argument is required.')
raise SystemExit(1) raise SystemExit(1)
setupStats(driver, url) # Use the loaded plugin
driver = setup_plugin.setup_stats(driver, url)
logger.log(logging.INFO, 'Starting server collector.') socket_url = firstValid(args.socket_url, os.getenv('SOCKET_URL'), default='localhost')
httpd = HTTPServer(('', 9092), partial(Handler, downloadStats, driver, logger)) try:
logger.info('Server collector started.') socket_port = int(firstValid(args.socket_port, os.getenv('SOCKET_PORT'), default=8094))
except ValueError:
logger.error('Invalid socket port provided. Exiting.')
raise SystemExit(1)
logger.info('Starting server collector.')
httpd = HTTPServer(('', 9092), partial(Handler, download_plugin.download_stats, driver, logger, socket_url, socket_port))
httpd.serve_forever() httpd.serve_forever()

55
monolith-entrypoint.sh Normal file
View File

@@ -0,0 +1,55 @@
#!/bin/bash
if [ -z "$TELEGRAF_HOSTNAME" ]; then
echo "Error: TELEGRAF_HOSTNAME is not set"
exit 1
fi
if [ -z "$TELEGRAF_MONGODB_DSN" ]; then
echo "Error: TELEGRAF_MONGODB_DSN is not set"
exit 1
fi
if [ -z "$TELEGRAF_MONGODB_DATABASE" ]; then
echo "Error: TELEGRAF_MONGODB_DATABASE is not set"
exit 1
fi
if [ -z "$SOCKET_PORT" ]; then
echo "Error: SOCKET_PORT is not set"
exit 1
fi
if [ -z "$VIDEO_URL" ]; then
echo "Error: VIDEO_URL is not set"
exit 1
fi
# Set the environment variables
export DSN=$TELEGRAF_MONGODB_DSN
export DATABASE=$TELEGRAF_MONGODB_DATABASE
export HOSTNAME=$TELEGRAF_HOSTNAME
# Start the Selenium hub
/opt/bin/entry_point.sh > /dev/null 2>&1 &
# Wait for Selenium hub to be ready
printf 'Waiting for Selenium standalone to be ready'
timeout=30
while ! curl -sSL "http://localhost:4444/wd/hub/status" 2>/dev/null | jq -e '.value.ready' | grep -q true; do
printf '.'
sleep 1
((timeout--))
if [ $timeout -le 0 ]; then
echo "Error: Selenium standalone did not become ready in time. Exiting..."
exit 1
fi
done
printf '\n'
# Start the Telegraf agent
telegraf --config ./telegraf.conf &
# Start the main Python script as PID 1
exec ./venv/bin/python main.py

31
plugins/example_plugin.py Normal file
View File

@@ -0,0 +1,31 @@
import logging
from selenium import webdriver
from selenium.webdriver.remote.webdriver import WebDriver as Remote
from utils.plugins_base import StatsSetupPlugin, StatsDownloadPlugin
logger = logging.getLogger(__name__)
class ExampleStatsSetupPlugin(StatsSetupPlugin):
def setup_stats(self, driver: webdriver.Chrome, url: str, retries: int = 5) -> webdriver.Chrome:
logger.info("Running ExampleStatsSetupPlugin...")
# Here you would implement the custom logic to setup stats
# For example, you could click on a button to display stats.
# You could also wait for an element to appear before continuing.
# This is just an example
driver.get(url)
return driver
class ExampleStatsDownloadPlugin(StatsDownloadPlugin):
def download_stats(self, driver: webdriver.Chrome, peersDict: dict, socket_url: str, socket_port: int):
logger.info("Running ExampleStatsDownloadPlugin...")
stats = {'message': 'Hello from ExampleStatsDownloadPlugin'}
# Here you would implement the custom logic to download stats
# and send them to the socket.
# This is just an example
print(f"Sending stats: {stats} to {socket_url}:{socket_port}")
# Remember to call the saveStats method to send the stats to the socket
super().saveStats([stats], socket_url, socket_port)

View File

@@ -1,2 +1,3 @@
selenium selenium
beautifulsoup4 beautifulsoup4
yaspin

30
server/README.md Normal file
View File

@@ -0,0 +1,30 @@
# Server
The repository contains a `server` directory with a simple MongoDB server (with initializations scripts) and WebUI that serves the WebRTC stats collected by the collector.
It's not mandatory to run and use this service, it's provided just as an example of how to store collected data.
## Setup
1. Change to the `server` directory:
```sh
cd server
```
2. Create and configure the environment file based on the `.env.example` file:
```sh
cp .env.example .env
```
3. Start the Docker containers:
```sh
docker compose up
```
The WebUI control panel will be available at [http://localhost:8081](http://localhost:8081).
# Credits
- [WebRTC Internals Exporter](https://github.com/vpalmisano/webrtc-internals-exporter)
- [WebRTC debugging with Prometheus/Grafana](https://medium.com/@vpalmisano/webrtc-debugging-with-prometheus-grafana-254b6ac71063)
- [MongoDB Docker Compose examples](https://github.com/TGITS/docker-compose-examples/tree/main/mongodb-docker-compose-examples)

View File

@@ -7,7 +7,7 @@
dedup_interval = "600s" dedup_interval = "600s"
[[inputs.socket_listener]] [[inputs.socket_listener]]
service_address = "udp://:8094" service_address = "udp://:${SOCKET_PORT}"
data_format = "xpath_json" data_format = "xpath_json"
[[inputs.socket_listener.xpath]] [[inputs.socket_listener.xpath]]
metric_name = "'peertube'" metric_name = "'peertube'"

5
utils/Convenience.py Normal file
View File

@@ -0,0 +1,5 @@
def firstValid(*args, default):
for arg in args:
if arg is not None:
return arg
return default

View File

@@ -3,10 +3,12 @@ import logging
from http.server import BaseHTTPRequestHandler from http.server import BaseHTTPRequestHandler
class Handler(BaseHTTPRequestHandler): class Handler(BaseHTTPRequestHandler):
def __init__(self, custom_func, driver, logger, *args, **kwargs): def __init__(self, custom_func, driver, logger, socket_url, socket_port, *args, **kwargs):
self._custom_func = custom_func self._custom_func = custom_func
self.logger = logger self.logger = logger
self.driver = driver self.driver = driver
self._socket_url = socket_url
self._socket_port = socket_port
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
def do_POST(self): def do_POST(self):
@@ -14,7 +16,7 @@ class Handler(BaseHTTPRequestHandler):
content_length = int(self.headers['Content-Length']) content_length = int(self.headers['Content-Length'])
post_data = self.rfile.read(content_length) post_data = self.rfile.read(content_length)
self.logger.log(logging.DEBUG, f"POST request,\nPath: {self.path}\nHeaders:\n{self.headers}\n\nBody:\n{post_data.decode('utf-8')}") self.logger.log(logging.DEBUG, f"POST request,\nPath: {self.path}\nHeaders:\n{self.headers}\n\nBody:\n{post_data.decode('utf-8')}")
self._custom_func(self.driver, json.loads(post_data.decode('utf-8'))) self._custom_func(self.driver, json.loads(post_data.decode('utf-8')), self._socket_url, self._socket_port)
self.send_response(200) self.send_response(200)
self.end_headers() self.end_headers()
self.wfile.write(b'POST request received') self.wfile.write(b'POST request received')
@@ -24,6 +26,11 @@ class Handler(BaseHTTPRequestHandler):
self.wfile.write(b'404 Not Found') self.wfile.write(b'404 Not Found')
def do_GET(self): def do_GET(self):
self.send_response(404) if self.path == '/heartbeat':
self.end_headers() self.send_response(200)
self.wfile.write(b'404 Not Found') self.end_headers()
self.wfile.write(b'Heartbeat OK')
else:
self.send_response(404)
self.end_headers()
self.wfile.write(b'404 Not Found')

29
utils/plugins_base.py Normal file
View File

@@ -0,0 +1,29 @@
import abc
import json
import socket
import logging
from selenium import webdriver
logger = logging.getLogger(__name__)
# Abstract Base Classes for Plugins
class StatsSetupPlugin(abc.ABC):
@abc.abstractmethod
def setup_stats(self, driver: webdriver.Remote | webdriver.Chrome, url: str, retries: int = 5) -> webdriver.Remote | webdriver.Chrome:
pass
class StatsDownloadPlugin(abc.ABC):
@abc.abstractmethod
def download_stats(self, driver: webdriver.Remote | webdriver.Chrome, peersDict: dict, socket_url: str, socket_port: int):
pass
@staticmethod
def saveStats(stats: list, socket_url: str, socket_port: int):
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
logger.debug(f'Saving stats: {json.dumps(stats, indent=4)}')
sock.sendto(json.dumps(stats).encode(), (socket_url, socket_port))
sock.close()
logger.debug('Sent stats to socket.')
except socket.error as e:
logger.error(f'Got socket error: {e}')

File diff suppressed because one or more lines are too long

View File

@@ -6,17 +6,17 @@ function log(...args) {
log("loaded"); log("loaded");
import "/assets/pako.min.js"; import "./assets/pako.min.js";
const DEFAULT_OPTIONS = { const DEFAULT_OPTIONS = {
url: "http://localhost:9092", url: process.env.WEBRTC_INTERNALS_EXPORTER_URL + ":9092",
username: "", username: "",
password: "", password: "",
updateInterval: 2, updateInterval: 2,
gzip: false, gzip: false,
job: "webrtc-internals-exporter", job: "webrtc-internals-exporter",
enabledOrigins: { }, enabledOrigins: {},
enabledStats: ["data-channel", "local-candidate", "remote-candidate"] enabledStats: ["data-channel", "local-candidate", "remote-candidate", "candidate-pair"]
}; };
const options = {}; const options = {};

View File

@@ -40,7 +40,7 @@
} }
], ],
"background": { "background": {
"service_worker": "background.js", "service_worker": "background.bundle.js",
"type": "module" "type": "module"
}, },
"web_accessible_resources": [ "web_accessible_resources": [

View File

@@ -22,7 +22,7 @@ class WebrtcInternalExporter {
}); });
window.postMessage({ event: "webrtc-internal-exporter:ready" }); window.postMessage({ event: "webrtc-internal-exporter:ready" });
this.collectAllStats(); setInterval(() => this.collectAndPostAllStats(), this.updateInterval);
} }
randomId() { randomId() {
@@ -40,6 +40,9 @@ class WebrtcInternalExporter {
pc.iceCandidateErrors = []; pc.iceCandidateErrors = [];
this.peerConnections.set(id, pc); this.peerConnections.set(id, pc);
pc.addEventListener("connectionstatechange", () => { pc.addEventListener("connectionstatechange", () => {
log(`connectionStateChange: ${pc.connectionState}`);
this.collectAndPostAllStats();
if (pc.connectionState === "closed") { if (pc.connectionState === "closed") {
this.peerConnections.delete(id); this.peerConnections.delete(id);
} }
@@ -83,24 +86,27 @@ class WebrtcInternalExporter {
} }
async collectAndPostSingleStat(id) { async collectAndPostSingleStat(id) {
const stats = await this.collectStats(id, this.collectAndPostSingleStat); const stats = await this.collectStats(id);
if (Object.keys(stats).length === 0 || !stats) return; if (Object.keys(stats).length === 0 || !stats) return;
window.postMessage( window.postMessage(
{ {
event: "webrtc-internal-exporter:peer-connection-stats", event: "webrtc-internal-exporter:peer-connection-stats",
stats stats: [stats]
}, },
stats [stats]
); );
log(`Single stat collected:`, [stats]);
} }
async collectAllStats() { async collectAndPostAllStats() {
const stats = []; const stats = [];
for (const [id, pc] of this.peerConnections) { for (const [id] of this.peerConnections) {
if (this.url && this.enabled) { if (this.url && this.enabled) {
const pcStats = await this.collectStats(id, pc); const pcStats = await this.collectStats(id);
if (Object.keys(pcStats).length === 0 || !pcStats) continue;
stats.push(pcStats); stats.push(pcStats);
} }
} }
@@ -108,28 +114,24 @@ class WebrtcInternalExporter {
window.postMessage( window.postMessage(
{ {
event: "webrtc-internal-exporter:peer-connections-stats", event: "webrtc-internal-exporter:peer-connections-stats",
data: JSON.parse(JSON.stringify(stats)), data: stats
}, },
stats
); );
log(`Stats collected:`, JSON.parse(JSON.stringify(stats))); log(`Stats collected:`, stats);
setTimeout(this.collectAllStats.bind(this), this.updateInterval);
return stats; return stats;
} }
/** /**
* @param {string} id * @param {string} id
* @param {RTCPeerConnection} pc
* @param {Function} binding
*/ */
async collectStats(id, pc, binding) { async collectStats(id) {
var completeStats = {}; var pc = this.peerConnections.get(id);
if (!pc) return;
if (!pc) { var completeStats = {};
pc = this.peerConnections.get(id);
if (!pc) return;
}
if (this.url && this.enabled) { if (this.url && this.enabled) {
try { try {
@@ -157,10 +159,6 @@ class WebrtcInternalExporter {
if (pc.connectionState === "closed") { if (pc.connectionState === "closed") {
this.peerConnections.delete(id); this.peerConnections.delete(id);
} else {
if (binding) {
setTimeout(binding.bind(this), this.updateInterval, id);
}
} }
return completeStats; return completeStats;

View File

@@ -0,0 +1,3 @@
module.exports = {
shouldPrintComment: () => false
};

View File

@@ -0,0 +1,26 @@
const { execSync } = require('child_process');
const args = process.argv.slice(2);
let url = '';
args.forEach((arg, index) => {
if (arg === '-u' || arg === '--url') {
url = args[index + 1];
} else if (arg === '-h' || arg === '--help') {
console.log('Usage: npm run build -- [-u|--url <url>]');
console.log('Options:');
console.log(' -u, --url <url> URL to use for the extension collector server');
console.log(' -h, --help Display this help message');
process.exit(0);
} else if (arg.startsWith('-')) {
console.error(`Unrecognized argument: ${arg}`);
process.exit(1);
}
});
if (url) {
console.log(`Building with URL: ${url}`);
execSync(`webpack --env URL=${url}`, { stdio: 'inherit' });
} else {
execSync('webpack', { stdio: 'inherit' });
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,33 @@
{
"name": "webrtc-internals-exporter",
"version": "1.0.0",
"private": true,
"scripts": {
"build": "node build.js"
},
"keywords": [],
"author": "Mirko Milovanovic",
"license": "MIT",
"devDependencies": {
"babel-loader": "^8.2.2",
"clean-webpack-plugin": "^3.0.0",
"copy-webpack-plugin": "^12.0.2",
"dotenv": "^16.4.7",
"file-loader": "^6.2.0",
"html-webpack-plugin": "^5.3.1",
"mini-css-extract-plugin": "^1.6.0",
"postcss": "^8.2.14",
"postcss-loader": "^5.2.0",
"postcss-preset-env": "^10.1.4",
"sass": "^1.32.12",
"sass-loader": "^11.0.1",
"serve": "^14.2.4",
"style-loader": "^2.0.0",
"terser-webpack-plugin": "^5.1.1",
"ts-loader": "^9.1.2",
"typescript": "^4.2.4",
"webpack": "^5.38.1",
"webpack-cli": "^4.7.2",
"webpack-dev-server": "^5.2.0"
}
}

View File

@@ -0,0 +1,34 @@
const path = require('path');
const { EnvironmentPlugin } = require('webpack');
const envPath = path.resolve(__dirname, '../../.env');
const envConfig = require('dotenv').config({ path: envPath }).parsed;
module.exports = (env) => {
const url = env.URL || 'http://localhost';
return {
entry: '../background.js',
target: 'web',
mode: 'production',
module: {
rules: [
{
test: /\.js?$/,
use: 'babel-loader',
exclude: /node_modules/,
},
],
},
resolve: { extensions: ['.tsx', '.ts', '.js'] },
output: {
filename: 'background.bundle.js',
path: path.resolve(__dirname, '../'),
publicPath: '',
},
plugins: [
new EnvironmentPlugin({
WEBRTC_INTERNALS_EXPORTER_URL: envConfig.WEBRTC_INTERNALS_EXPORTER_URL || url
}),
],
};
};