Compare commits

..

No commits in common. "main" and "0.1.0-c.1" have entirely different histories.

19 changed files with 84 additions and 209 deletions

View file

@ -56,37 +56,3 @@ jobs:
run: pip install twine run: pip install twine
- name: Upload package to registry - name: Upload package to registry
run: python -m twine upload --repository-url ${{ secrets.REPOSITORY_URL }} -u ${{ secrets.TWINE_DEPLOY_USER }} -p ${{ secrets.TWINE_DEPLOY_PASSWORD }} dist/* run: python -m twine upload --repository-url ${{ secrets.REPOSITORY_URL }} -u ${{ secrets.TWINE_DEPLOY_USER }} -p ${{ secrets.TWINE_DEPLOY_PASSWORD }} dist/*
build-and-push-container:
needs: [ "publish-artifacts" ]
runs-on: dind
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log into Docker Package Registry
uses: docker/login-action@v3
with:
registry: forgejo.neshweb.net
username: ${{ secrets.FORGEJO_USERNAME }}
password: ${{ secrets.FORGEJO_TOKEN }}
- name: Build and push to Docker Package Registry
uses: docker/build-push-action@v5
with:
build-args: |
PACKAGE_VERSION=${{ github.ref_name }}
push: true
tags: forgejo.neshweb.net/firq/dockge-cli:${{ github.ref_name }}
release:
needs: [ build-and-push-container, publish-artifacts ]
if: success()
runs-on: docker
steps:
- name: Release New Version
uses: actions/forgejo-release@v1
with:
direction: upload
url: https://forgejo.neshweb.net
release-dir: release
token: ${{ secrets.FORGEJO_TOKEN }}
tag: ${{ github.ref_name }}

View file

@ -1,4 +0,0 @@
FROM forgejo.neshweb.net/ci-docker-images/python-neshweb:3.11
ARG PACKAGE_VERSION=0.1.0
RUN pip install dockge-cli==${PACKAGE_VERSION}

View file

@ -1,67 +1,3 @@
# dockge-cli # dockge-cli
A simple CLI application written in Python for communicating with Dockge using websockets A simple CLI application written in Python for communicating with Dockge using websockets
## Background
Dockge (spoken dock-ge or dockage) is a tool to manage docker-compose stacks from a web ui. It is developed by louislam, who also develops UptimeKuma.
Dockge itself doesn't offer any kind of API or programmatic access, as it is just intended for managing stacks via UI.
My current deployment solution for firq.dev and fgo-ta.com is based on Dockge, and I was over it always having to reload the stack whenever I pushed an update. Instead, I wanted to have this as a separate CI step, automatically redeploying a givens stack.
As Dockge is using a websocket-based system under the hood, it was easy to take a look at how communication occurs. In general, communication is achieved by leveraging socket.io for the data. Since Python already offers a solution for socket.io, it is just a matter of emulating the calls the webui sends and receives.
In the end, this is the current result that works pretty well for my understanding. I am still trying to improve upon some issues (login times out, stability, features), but in general this works as a fine solution for automatic stack updating.
## Installation
Install it from the custom package index using
```shell
pip install --extra-index-url https://forgejo.neshweb.net/api/packages/Firq/pypi/simple/ dockge-cli
```
Alternativly, install it using this repository. When installing for development, make sure to install with the additional dependencies
```shell
pip install -e .[lint,typing]
```
## Usage
Call the CLI using `dockge-cli` or `dockge`.
```shell
usage: dockge_cli [-h] [--version] {host,login,logout,list,status,restart,start,stop,down,update,exit,help}
CLI interface for interacting with Dockge
positional arguments:
{host,login,logout,list,status,restart,start,stop,down,update,exit,help}
options:
-h, --help show this help message and exit
--version show program's version number and exit
```
Help for each individual command can be invoked by calling `dockge-cli help <command>`
## The magic behind this
Generally, this makes use of the underlying Websockets API that the Dockge frontend uses to communicate with the server. By analyzing the traffic and looking into the codebase, I was able to reverse most of the packets that are being sent. This allows me to then contruct, send and receive my own packets, making the whole thing work.
There are some things that need to be taken into account for this: For one, dockge uses socket.io for the websocket communication. This meant I had to find the corresponding socket.io version to get the correct version of python-socketio. In addition, I had to find out how the authorization mechanism behind this works.
After finishing up the first prototype, the workings are as follows:
1. A websocket session is established using socket.io - this happens automatically
2. After the session is ready, the `login` command is sent together with a provided username and password
3. Once the CLI is authorized, the selected command is sent
4. The CLI waits for any response values and exits once the command has executed successfully
To provide a smooth experience, both the credentials and the remote host URI are stored on disk. just like the `docker` cli, the credentials are not encrypted, meaning it is advised to either clear the credentials after use OR to use the `--username` and `--password` parameters. This is especially recommended for CI applications.
## Known issues
This CLI does not work when Mullvad is used, as Mullvad actively blocks port forwarding (which python-socketio uses)

1
dockge_cli/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
.temp/*

View file

@ -1,22 +1,18 @@
from urllib.parse import urlparse from urllib.parse import urlparse
from getpass import getpass from getpass import getpass
import re
from ...models import Credentials from ...models import Credentials
from ...service import storage from ...service import storage
from ...service.connection import DockgeConnection from ...service.communicate import DockgeConnection
from ..utils import stack_formatter, status_formatter, generic_formatter, credential_parser_factory from ..utils import stack_formatter, status_formatter, generic_formatter, get_credential_parser
class FunctionBindings(): class ExecutionCommands():
""" """
Helper class that provides all the static methods in an organized way Helper class that provides all the static methods in an organized way
This is an abstraction layer of the CLI, as those functions only do little preprocessing before calling the actural DockgeConnection This is an abstraction layer of the CLI, as those functions only do little preprocessing before calling the actural DockgeConnection
""" """
@staticmethod @staticmethod
def __setup(): def __setup():
"""
Creates a connection and logs into Dockge
"""
con = DockgeConnection() con = DockgeConnection()
con.connect_and_login() con.connect_and_login()
return con return con
@ -27,12 +23,10 @@ class FunctionBindings():
host command binding host command binding
""" """
if len(extra_args) > 0: if len(extra_args) > 0:
mat = re.search(r"((\w+\.)?\w+\.\w+(\/.+)?)", extra_args[0], re.IGNORECASE) res = urlparse(extra_args[0])
if mat is None:
raise ValueError("Given host did not match regex")
res = urlparse(f"https://{mat[0]}")
if all([res.scheme, res.netloc]): if all([res.scheme, res.netloc]):
storage.put("host", mat[0]) host = extra_args[0].rstrip("/").replace("https://", "").replace("wss://", "")
storage.put("host", host)
else: else:
raise ValueError(f"Malformed URL {extra_args[0]}") raise ValueError(f"Malformed URL {extra_args[0]}")
print(storage.get("host")) print(storage.get("host"))
@ -42,9 +36,8 @@ class FunctionBindings():
""" """
login command binding login command binding
""" """
print(f"WARNING! These credentials will be saved unencrypted in {storage._file.absolute()}")
if len(extra_args) > 0: if len(extra_args) > 0:
credentials = credential_parser_factory().parse_args(extra_args, namespace=Credentials) credentials = get_credential_parser().parse_args(extra_args, namespace=Credentials)
storage.put("username", credentials.username, encoded=True) storage.put("username", credentials.username, encoded=True)
storage.put("password", credentials.password, encoded=True) storage.put("password", credentials.password, encoded=True)
return return
@ -71,7 +64,7 @@ class FunctionBindings():
""" """
list command binding list command binding
""" """
con = FunctionBindings.__setup() con = ExecutionCommands.__setup()
stack_formatter(con.list_stacks()) stack_formatter(con.list_stacks())
con.disconnect() con.disconnect()
@ -80,7 +73,9 @@ class FunctionBindings():
""" """
status command binding status command binding
""" """
con = FunctionBindings.__setup() if extra_args is None:
raise ValueError
con = ExecutionCommands.__setup()
status_formatter(con.list_stack(extra_args[0])) status_formatter(con.list_stack(extra_args[0]))
con.disconnect() con.disconnect()
@ -89,7 +84,9 @@ class FunctionBindings():
""" """
restart command binding restart command binding
""" """
con = FunctionBindings.__setup() if extra_args is None:
raise ValueError
con = ExecutionCommands.__setup()
generic_formatter(con.restart(extra_args[0])) generic_formatter(con.restart(extra_args[0]))
con.disconnect() con.disconnect()
@ -98,7 +95,9 @@ class FunctionBindings():
""" """
update command binding update command binding
""" """
con = FunctionBindings.__setup() if extra_args is None:
raise ValueError
con = ExecutionCommands.__setup()
generic_formatter(con.update(extra_args[0])) generic_formatter(con.update(extra_args[0]))
con.disconnect() con.disconnect()
@ -107,7 +106,9 @@ class FunctionBindings():
""" """
stop command binding stop command binding
""" """
con = FunctionBindings.__setup() if extra_args is None:
raise ValueError
con = ExecutionCommands.__setup()
generic_formatter(con.stop(extra_args[0])) generic_formatter(con.stop(extra_args[0]))
con.disconnect() con.disconnect()
@ -116,7 +117,9 @@ class FunctionBindings():
""" """
start command binding start command binding
""" """
con = FunctionBindings.__setup() if extra_args is None:
raise ValueError
con = ExecutionCommands.__setup()
generic_formatter(con.start(extra_args[0])) generic_formatter(con.start(extra_args[0]))
con.disconnect() con.disconnect()
@ -125,7 +128,9 @@ class FunctionBindings():
""" """
down command binding down command binding
""" """
con = FunctionBindings.__setup() if extra_args is None:
raise ValueError
con = ExecutionCommands.__setup()
generic_formatter(con.down(extra_args[0])) generic_formatter(con.down(extra_args[0]))
con.disconnect() con.disconnect()

View file

@ -1,6 +1,6 @@
from typing import List from typing import List
from ...models import Command from ...models import Command
from .functions import FunctionBindings from .bindings import ExecutionCommands
mapping: List[Command] = [ mapping: List[Command] = [
Command( Command(
@ -8,83 +8,83 @@ mapping: List[Command] = [
description="Sets and gets the URI of the dockge instance. Remove any unnecessary subdomains/protocols from the URI", description="Sets and gets the URI of the dockge instance. Remove any unnecessary subdomains/protocols from the URI",
args=1, args=1,
optional=True, optional=True,
func=FunctionBindings.host bind=ExecutionCommands.host
), ),
Command( Command(
cmd="login", cmd="login",
description="Logs into a given dockge account, either with an interactive dialogue or by passing --user and --password", description="Logs into a given dockge account, either with an interactive dialogue or by passing --user and --password",
args=2, args=2,
optional=True, optional=True,
func=FunctionBindings.login bind=ExecutionCommands.login
), ),
Command( Command(
cmd="logout", cmd="logout",
description="Removes the credentials from the local storage.", description="Removes the credentials from the local storage.",
args=0, args=0,
optional=False, optional=False,
func=FunctionBindings.logout bind=ExecutionCommands.logout
), ),
Command( Command(
cmd="list", cmd="list",
description="Lists all available stacks with their status", description="Lists all available stacks with their status",
args=0, args=0,
optional=False, optional=False,
func=FunctionBindings.list bind=ExecutionCommands.list
), ),
Command( Command(
cmd="status", cmd="status",
description="Returns the status of one stack", description="Returns the status of one stack",
args=1, args=1,
optional=False, optional=False,
func=FunctionBindings.status bind=ExecutionCommands.status
), ),
Command( Command(
cmd="restart", cmd="restart",
description="Restarts a given stack", description="Restarts a given stack",
args=1, args=1,
optional=False, optional=False,
func=FunctionBindings.restart bind=ExecutionCommands.restart
), ),
Command( Command(
cmd="start", cmd="start",
description="Starts a given stack", description="Starts a given stack",
args=1, args=1,
optional=False, optional=False,
func=FunctionBindings.start bind=ExecutionCommands.start
), ),
Command( Command(
cmd="stop", cmd="stop",
description="Stops a given stack", description="Stops a given stack",
args=1, args=1,
optional=False, optional=False,
func=FunctionBindings.stop bind=ExecutionCommands.stop
), ),
Command( Command(
cmd="down", cmd="down",
description="Stop & Downs a given stack", description="Stop & Downs a given stack",
args=1, args=1,
optional=False, optional=False,
func=FunctionBindings.down bind=ExecutionCommands.down
), ),
Command( Command(
cmd="update", cmd="update",
description="Updates a stack", description="Updates a stack",
args=1, args=1,
optional=False, optional=False,
func=FunctionBindings.update bind=ExecutionCommands.update
), ),
Command( Command(
cmd="exit", cmd="exit",
description="Exits the CLI - this will reset all settings, including credentials and host", description="Exits the CLI - this will reset all settings, including credentials and host",
args=0, args=0,
optional=False, optional=False,
func=FunctionBindings.exit bind=ExecutionCommands.exit
), ),
Command( Command(
cmd="help", cmd="help",
description="Displays helping hints for commands", description="Displays helping hints for commands",
args=1, args=1,
optional=True, optional=True,
func=FunctionBindings.help bind=ExecutionCommands.help
) )
] ]

View file

@ -0,0 +1,7 @@
from ...models import Command
from .descriptors import mapping
commands: dict[str, Command] = {}
for descriptor in mapping:
commands.update({ descriptor.cmd: descriptor })

View file

@ -1,4 +0,0 @@
from ...models import Command
from .mappings import mapping
commands: dict[str, Command] = { c.cmd: c for c in mapping }

View file

@ -3,7 +3,7 @@ import sys
from .. import __version__ from .. import __version__
from ..models import Arguments from ..models import Arguments
from .commands import commands from .commandprovider.factory import commands
def parse_arguments(): def parse_arguments():
""" """

View file

@ -1,4 +1,4 @@
from .commands import commands from .commandprovider.factory import commands
def display_help(extra_args): def display_help(extra_args):
""" """
@ -32,4 +32,4 @@ def run(command, args):
display_help(args) display_help(args)
return return
c.func(args) c.bind(args)

View file

@ -2,7 +2,7 @@ import argparse
from tabulate import tabulate from tabulate import tabulate
from ..models import StackStatus from ..models import StackStatus
def credential_parser_factory(): def get_credential_parser():
""" """
Creates a new parser for login credentials Creates a new parser for login credentials
""" """
@ -23,7 +23,7 @@ def stack_formatter(stacks):
table, headers = [], ["Stackname", "Status"] table, headers = [], ["Stackname", "Status"]
for key, val in stacks["stackList"].items(): for key, val in stacks["stackList"].items():
table.append([key, StackStatus(val['status']).name.lower()]) table.append([key, StackStatus(val['status']).name])
print(tabulate(table, headers=headers, tablefmt="github"), "\n") print(tabulate(table, headers=headers, tablefmt="github"), "\n")

View file

@ -1,3 +1,3 @@
from .codes import StackStatus from .codes import StackStatus
from .command import Command from .commands import Command
from .parser import Arguments, Credentials from .parser import Arguments, Credentials

View file

@ -2,8 +2,9 @@ from enum import Enum
class StackStatus(Enum): class StackStatus(Enum):
""" """
mapping for plaintext vs statuscode mapping codes for status vs text
""" """
INACTIVE = 1 # pylint: disable=invalid-name
RUNNING = 3 inactive = 1
EXITED = 4 running = 3
exited = 4

View file

@ -6,7 +6,7 @@ class Command(BaseModel):
Basic command structure for the CLI to automatically generate valid commands Basic command structure for the CLI to automatically generate valid commands
""" """
cmd: str cmd: str
func: Callable bind: Callable
args: int args: int
optional: bool optional: bool
description: str description: str

View file

@ -1 +0,0 @@
.storage/*

View file

@ -32,7 +32,7 @@ class DockgeConnection:
def _init_events(self): def _init_events(self):
@self._sio.event @self._sio.event
def connect(): def connect():
self.login() self.connect()
print("Connected!") print("Connected!")
@self._sio.event @self._sio.event
@ -63,7 +63,6 @@ class DockgeConnection:
success = True success = True
else: else:
print("Issue with login procedure") print("Issue with login procedure")
print(data)
return success return success
# Functions # Functions
@ -71,11 +70,10 @@ class DockgeConnection:
""" """
Connect to the websocket Connect to the websocket
""" """
# Dockge uses Socket.io for the websockets, so this URI and params are always the same self._sio.connect(f"https://{self._host}/socket.io/", transports=['websocket'])
self._sio.connect(f"https://{self._host}/socket.io/") self.connect()
self.login()
def login(self): def connect(self):
""" """
Log into dockge using basicauth Log into dockge using basicauth
Retries 5 times when timeouts occur Retries 5 times when timeouts occur
@ -86,11 +84,6 @@ class DockgeConnection:
data = None data = None
retry, count = True, 0 retry, count = True, 0
if not storage.exists("username"):
raise ValueError("Missing username")
if not storage.exists("password"):
raise ValueError("Missing password")
while retry and count < 5: while retry and count < 5:
try: try:
data = self._sio.call( data = self._sio.call(
@ -98,13 +91,12 @@ class DockgeConnection:
{ {
"username": storage.get("username", encoded=True), "username": storage.get("username", encoded=True),
"password": storage.get("password", encoded=True), "password": storage.get("password", encoded=True),
"token": "" "token":""
}, },
timeout=10 timeout=5
) )
retry = False retry = False
except socketio.exceptions.TimeoutError: except socketio.exceptions.TimeoutError:
print("Reached timeout for login, retrying ...")
retry = True retry = True
count += 1 count += 1
@ -127,42 +119,42 @@ class DockgeConnection:
""" """
Lists status for a stack Lists status for a stack
""" """
ret = self._sio.call("agent", ("", "serviceStatusList", name), timeout=10) ret = self._sio.call("agent", ("", "serviceStatusList", name), timeout=5)
return ret return ret
def restart(self, name): def restart(self, name):
""" """
Restarts a given stack Restarts a given stack
""" """
ret = self._sio.call("agent", ("", "restartStack", name), timeout=30) ret = self._sio.call("agent", ("", "restartStack", name), timeout=10)
return ret return ret
def update(self, name): def update(self, name):
""" """
Updates a given stack Updates a given stack
""" """
ret = self._sio.call("agent", ("", "updateStack", name), timeout=30) ret = self._sio.call("agent", ("", "updateStack", name), timeout=10)
return ret return ret
def stop(self, name): def stop(self, name):
""" """
Stops a given stack Stops a given stack
""" """
ret = self._sio.call("agent", ("", "stopStack", name), timeout=30) ret = self._sio.call("agent", ("", "stopStack", name), timeout=10)
return ret return ret
def start(self, name): def start(self, name):
""" """
Starts a given stack Starts a given stack
""" """
ret = self._sio.call("agent", ("", "startStack", name), timeout=30) ret = self._sio.call("agent", ("", "startStack", name), timeout=10)
return ret return ret
def down(self, name): def down(self, name):
""" """
Stops and downs a given stack Stops and downs a given stack
""" """
ret = self._sio.call("agent", ("", "downStack", name), timeout=30) ret = self._sio.call("agent", ("", "downStack", name), timeout=10)
return ret return ret
def disconnect(self): def disconnect(self):

View file

@ -3,44 +3,28 @@ import pathlib
import base64 import base64
import yaml import yaml
_storagepath = pathlib.Path(__file__).parent / ".storage" _storagepath = pathlib.Path(__file__).parents[1] / ".temp"
_storagepath.mkdir(exist_ok=True, parents=True)
_file = _storagepath / "storage.yaml" _file = _storagepath / "storage.yaml"
def create_file_when_missing(): _storagepath.mkdir(exist_ok=True, parents=True)
def fileexists():
""" """
Checks if storage file does exist, creates it when necessary Checks if storage file does exist, creates it when necessary
""" """
if _file.exists():
return
with open(_file, 'a', encoding="utf-8"):
os.utime(_file, None)
def exists(key: str) -> bool:
"""
Checks if a given key exists in the storage file
"""
if not _file.exists(): if not _file.exists():
return False with open(_file, 'a', encoding="utf-8"):
os.utime(_file, None)
with open(_file, "r", encoding="utf-8") as file:
content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader)
return key in content
def put(key: str, value: str, encoded=False): def put(key: str, value: str, encoded=False):
""" """
Puts a given value with a given key into the storage file Puts a given value with a given key into the storage file
Encodes the data as base64 when encoded is set to true Encodes the data as base64 when encoded is set to true
""" """
if not _file.exists(): fileexists()
create_file_when_missing() with open(_file, "r+", encoding="utf-8") as file:
with open(_file, "r", encoding="utf-8") as file:
content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader) or {} content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader) or {}
content.update({ key: str(base64.b64encode(value.encode()), "utf-8") if encoded else value }) content.update({ key: str(base64.b64encode(value.encode())) if encoded else value })
with open(_file, "w+", encoding="utf-8") as file: with open(_file, "w+", encoding="utf-8") as file:
yaml.dump(content, file, Dumper=yaml.SafeDumper) yaml.dump(content, file, Dumper=yaml.SafeDumper)
@ -48,13 +32,10 @@ def remove(key: str):
""" """
Removed a given key from the storage file Removed a given key from the storage file
""" """
if not _file.exists(): fileexists()
create_file_when_missing()
with open(_file, "r", encoding="utf-8") as file: with open(_file, "r", encoding="utf-8") as file:
content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader) or {} content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader) or {}
content.pop(key, None) content.pop(key, None)
with open(_file, "w+", encoding="utf-8") as file: with open(_file, "w+", encoding="utf-8") as file:
yaml.dump(content, file, Dumper=yaml.SafeDumper) yaml.dump(content, file, Dumper=yaml.SafeDumper)
@ -66,19 +47,15 @@ def get(key: str, encoded=False):
value: str | None = None value: str | None = None
if not _file.exists(): if not _file.exists():
return None return None
with open(_file, "r", encoding="utf-8") as file: with open(_file, "r", encoding="utf-8") as file:
content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader) content: dict[str, str] = yaml.load(file, Loader=yaml.SafeLoader)
value = content.get(key, None) value = content.get(key, None)
if value is None: if value is None:
return None return None
return base64.b64decode(value.encode()).decode() if encoded else value return base64.b64decode(value).decode() if encoded else value
def clear(): def clear():
""" """
Deletes the storage file Deletes the storage file
""" """
if not _file.exists():
return
_file.unlink() _file.unlink()

View file

@ -1,10 +1,9 @@
[project] [project]
name = "dockge_cli" name = "dockge_cli"
version = "0.1.2" version = "0.1.0-c.1"
dependencies = [ dependencies = [
"pyyaml~=6.0.1", "pyyaml~=6.0.1",
"pydantic~=2.8.0", "pydantic~=2.8.0",
"requests~=2.32.3",
"python-socketio~=5.11.3", "python-socketio~=5.11.3",
"websocket-client~=1.8.0", "websocket-client~=1.8.0",
"tabulate ~=0.9.0", "tabulate ~=0.9.0",
@ -12,7 +11,7 @@ dependencies = [
requires-python = ">= 3.10" requires-python = ">= 3.10"
authors = [{name = "Firq", email = "firelp42@gmail.com"}] authors = [{name = "Firq", email = "firelp42@gmail.com"}]
maintainers = [{name = "Firq", email = "firelp42@gmail.com"}] maintainers = [{name = "Firq", email = "firelp42@gmail.com"}]
description = "CLI for interacting with dockge" description = "CLi for interacting with dockge"
classifiers = [ classifiers = [
"Development Status :: 3 - Alpha", "Development Status :: 3 - Alpha",
"Programming Language :: Python :: 3", "Programming Language :: Python :: 3",