feat: Logs filtering, and log queue with group concurrency

This commit is contained in:
jamesread 2026-06-16 14:16:43 +01:00
parent 37cde0d982
commit 6aa672e6c0
42 changed files with 2798 additions and 548 deletions

View File

@ -6,9 +6,17 @@
listenAddressSingleHTTPFrontend: 0.0.0.0:1337 listenAddressSingleHTTPFrontend: 0.0.0.0:1337
# Choose from INFO (default), WARN and DEBUG # Choose from INFO (default), WARN and DEBUG
# Docs: https://docs.olivetin.app/advanced_configuration/logs.html # Docs: https://docs.olivetin.app/advanced_configuration/logs.html
logLevel: "INFO" logLevel: "INFO"
# Action groups share a concurrency limit across multiple actions. When the
# limit is reached, additional requests are queued and run in order.
# Docs: https://docs.olivetin.app/action_customization/concurrency.html#action-groups
actionGroups:
backup-jobs:
maxConcurrent: 1
icon: backup
# Actions are commands that are executed by OliveTin, and normally show up as # Actions are commands that are executed by OliveTin, and normally show up as
# buttons on the WebUI. # buttons on the WebUI.
# #
@ -69,19 +77,26 @@ actions:
- "@hourly" - "@hourly"
# You are not limited to operating system commands, and of course you can run # You are not limited to operating system commands, and of course you can run
# your own scripts. Here `maxConcurrent` stops the script running multiple # your own scripts. The backup-jobs action group limits how many backup-related
# times in parallel. There is also a timeout that will kill the command if it # actions can run at once; extra requests are queued instead of blocked.
# runs for too long. # There is also a timeout that will kill the command if it runs for too long.
- title: Run backup script - title: Run backup script
shell: /opt/backupScript.sh shell: /opt/backupScript.sh
shellAfterCompleted: "apprise -t 'Notification: Backup script completed' -b 'The backup script completed with code {{ exitCode}}. The log is: \n {{ output }} '" shellAfterCompleted: "apprise -t 'Notification: Backup script completed' -b 'The backup script completed with code {{ exitCode}}. The log is: \n {{ output }} '"
maxConcurrent: 1 groups: [ backup-jobs ]
timeout: 10 timeout: 10
icon: backup icon: backup
popupOnStart: execution-dialog popupOnStart: execution-dialog
# https://docs.olivetin.app/action_execution/oncalendar.html # https://docs.olivetin.app/action_execution/oncalendar.html
execOnCalendarFile: examples/demo-olivetin-calendar.yaml execOnCalendarFile: examples/demo-olivetin-calendar.yaml
- title: Verify backup archive
shell: sleep 3 && echo "Backup archive verified"
groups: [ backup-jobs ]
timeout: 30
icon: backup
popupOnStart: execution-dialog
# When you want to prompt users for input, that is when you should use # When you want to prompt users for input, that is when you should use
# `arguments` - this presents a popup dialog and asks for argument values. # `arguments` - this presents a popup dialog and asks for argument values.
# #
@ -147,7 +162,7 @@ actions:
# #
# Docs: https://docs.olivetin.app/reference/reference_themes_for_users.html # Docs: https://docs.olivetin.app/reference/reference_themes_for_users.html
- title: Get OliveTin Theme - title: Get OliveTin Theme
exec: exec:
- "olivetin-get-theme" - "olivetin-get-theme"
- "{{ themeGitRepo }}" - "{{ themeGitRepo }}"
- "{{ themeFolderName }}" - "{{ themeFolderName }}"
@ -372,7 +387,7 @@ dashboards:
# Security - Authentication # Security - Authentication
# This setting effectively enables or disables guests. # This setting effectively enables or disables guests.
# If set to "true", then users will have to login to do anything. # If set to "true", then users will have to login to do anything.
authRequireGuestsToLogin: false authRequireGuestsToLogin: false
@ -381,7 +396,7 @@ authRequireGuestsToLogin: false
# and JWT authentication which are documented separately. # and JWT authentication which are documented separately.
# #
# Docs: https://docs.olivetin.app/security/local.html # Docs: https://docs.olivetin.app/security/local.html
# #
# How to get a hashed password: # How to get a hashed password:
# Docs: https://docs.olivetin.app/security/local.html#_get_a_argon2id_hashed_password # Docs: https://docs.olivetin.app/security/local.html#_get_a_argon2id_hashed_password
authLocalUsers: authLocalUsers:

View File

@ -3,7 +3,7 @@
By default, OliveTin will allow you to run several instances of an action at the same time. For example, an action might take 20 seconds, and if you click the button 3 times, for a time there will be 3 actions running at the same time. By default, OliveTin will allow you to run several instances of an action at the same time. For example, an action might take 20 seconds, and if you click the button 3 times, for a time there will be 3 actions running at the same time.
Sometimes you don't want to allow this - an example case where it would not make sense is in the case of a backup script. To stop this, we can set `maxConcurrent` to `1`. Sometimes you don't want to allow this - an example case where it would not make sense is in the case of a backup script. To stop this, we can set `maxConcurrent` to `1`.
[source,yaml] [source,yaml]
---- ----
@ -29,4 +29,30 @@ WARN Blocked from executing. This would mean this action is running 2 times conc
Naturally, you can set `maxConcurrent` to `3` or some other number, to limit the amount of times the action executes at once. Naturally, you can set `maxConcurrent` to `3` or some other number, to limit the amount of times the action executes at once.
== Action groups
Sometimes you need to limit concurrency across several different actions. For example, Unity only allows one build at a time, but you might have separate actions for different platforms.
Use `actionGroups` to define a shared limit, and assign actions to a group with `groups`:
[source,yaml]
----
actionGroups:
unity:
maxConcurrent: 1
actions:
- title: Unity Android Build
shell: /opt/unity/build-android.sh
groups: [ unity ]
- title: Unity iOS Build
shell: /opt/unity/build-ios.sh
groups: [ unity ]
----
When the group limit is reached, additional requests are queued automatically and run in order when a slot becomes free. Queued executions appear in the logs with a queued status.
Per-action `maxConcurrent` still applies separately. If the same action binding is started twice while one is already running, the second request is blocked immediately (not queued).
The queue is held in memory. If OliveTin restarts while actions are queued, those queued requests are not preserved.

View File

@ -30,6 +30,9 @@
"process": "^0.11.10", "process": "^0.11.10",
"stylelint": "^17.13.0", "stylelint": "^17.13.0",
"stylelint-config-standard": "^40.0.0" "stylelint-config-standard": "^40.0.0"
},
"engines": {
"node": ">=22.0.0"
} }
}, },
"node_modules/@babel/code-frame": { "node_modules/@babel/code-frame": {

View File

@ -38,5 +38,8 @@
"vue": "^3.5.38", "vue": "^3.5.38",
"vue-i18n": "^11.4.5", "vue-i18n": "^11.4.5",
"vue-router": "^5.1.0" "vue-router": "^5.1.0"
},
"engines": {
"node": ">=22.0.0"
} }
} }

View File

@ -568,6 +568,13 @@ export declare type GetLogsRequest = Message<"olivetin.api.v1.GetLogsRequest"> &
* @generated from field: int64 page_size = 3; * @generated from field: int64 page_size = 3;
*/ */
pageSize: bigint; pageSize: bigint;
/**
* Optional filter expression (see logs UI syntax help)
*
* @generated from field: string filter = 4;
*/
filter: string;
}; };
/** /**
@ -673,6 +680,16 @@ export declare type LogEntry = Message<"olivetin.api.v1.LogEntry"> & {
* @generated from field: string binding_id = 20; * @generated from field: string binding_id = 20;
*/ */
bindingId: string; bindingId: string;
/**
* @generated from field: bool queued = 21;
*/
queued: boolean;
/**
* @generated from field: string queued_for_group = 22;
*/
queuedForGroup: string;
}; };
/** /**
@ -774,6 +791,85 @@ export declare type GetActionLogsResponse = Message<"olivetin.api.v1.GetActionLo
*/ */
export declare const GetActionLogsResponseSchema: GenMessage<GetActionLogsResponse>; export declare const GetActionLogsResponseSchema: GenMessage<GetActionLogsResponse>;
/**
* @generated from message olivetin.api.v1.GetExecutionQueueRequest
*/
export declare type GetExecutionQueueRequest = Message<"olivetin.api.v1.GetExecutionQueueRequest"> & {
};
/**
* Describes the message olivetin.api.v1.GetExecutionQueueRequest.
* Use `create(GetExecutionQueueRequestSchema)` to create a new message.
*/
export declare const GetExecutionQueueRequestSchema: GenMessage<GetExecutionQueueRequest>;
/**
* @generated from message olivetin.api.v1.ExecutionQueueGroup
*/
export declare type ExecutionQueueGroup = Message<"olivetin.api.v1.ExecutionQueueGroup"> & {
/**
* @generated from field: string binding_id = 1;
*/
bindingId: string;
/**
* @generated from field: string action_title = 2;
*/
actionTitle: string;
/**
* @generated from field: string action_icon = 3;
*/
actionIcon: string;
/**
* @generated from field: int32 max_concurrent = 4;
*/
maxConcurrent: number;
/**
* @generated from field: int32 active_count = 5;
*/
activeCount: number;
/**
* @generated from field: string entity_prefix = 6;
*/
entityPrefix: string;
/**
* @generated from field: repeated olivetin.api.v1.LogEntry entries = 7;
*/
entries: LogEntry[];
};
/**
* Describes the message olivetin.api.v1.ExecutionQueueGroup.
* Use `create(ExecutionQueueGroupSchema)` to create a new message.
*/
export declare const ExecutionQueueGroupSchema: GenMessage<ExecutionQueueGroup>;
/**
* @generated from message olivetin.api.v1.GetExecutionQueueResponse
*/
export declare type GetExecutionQueueResponse = Message<"olivetin.api.v1.GetExecutionQueueResponse"> & {
/**
* @generated from field: repeated olivetin.api.v1.ExecutionQueueGroup groups = 1;
*/
groups: ExecutionQueueGroup[];
/**
* @generated from field: int32 total_active = 2;
*/
totalActive: number;
};
/**
* Describes the message olivetin.api.v1.GetExecutionQueueResponse.
* Use `create(GetExecutionQueueResponseSchema)` to create a new message.
*/
export declare const GetExecutionQueueResponseSchema: GenMessage<GetExecutionQueueResponse>;
/** /**
* @generated from message olivetin.api.v1.ValidateArgumentTypeRequest * @generated from message olivetin.api.v1.ValidateArgumentTypeRequest
*/ */
@ -1816,6 +1912,14 @@ export declare const OliveTinApiService: GenService<{
input: typeof GetActionLogsRequestSchema; input: typeof GetActionLogsRequestSchema;
output: typeof GetActionLogsResponseSchema; output: typeof GetActionLogsResponseSchema;
}, },
/**
* @generated from rpc olivetin.api.v1.OliveTinApiService.GetExecutionQueue
*/
getExecutionQueue: {
methodKind: "unary";
input: typeof GetExecutionQueueRequestSchema;
output: typeof GetExecutionQueueResponseSchema;
},
/** /**
* @generated from rpc olivetin.api.v1.OliveTinApiService.ValidateArgumentType * @generated from rpc olivetin.api.v1.OliveTinApiService.ValidateArgumentType
*/ */

File diff suppressed because one or more lines are too long

View File

@ -70,8 +70,8 @@ const showArgumentForm = ref(false)
const rateLimitExpires = ref(0) const rateLimitExpires = ref(0)
const isRateLimited = ref(false) const isRateLimited = ref(false)
const rateLimitMessage = ref('') const rateLimitMessage = ref('')
let rateLimitInterval = null const rateLimitInterval = ref(null)
let isComponentMounted = true const isComponentMounted = ref(true)
// Animation classes // Animation classes
const buttonClasses = ref([]) const buttonClasses = ref([])
@ -151,9 +151,9 @@ function updateRateLimitStatus() {
if (rateLimitExpires.value === 0) { if (rateLimitExpires.value === 0) {
isRateLimited.value = false isRateLimited.value = false
rateLimitMessage.value = '' rateLimitMessage.value = ''
if (rateLimitInterval) { if (rateLimitInterval.value) {
clearInterval(rateLimitInterval) clearInterval(rateLimitInterval.value)
rateLimitInterval = null rateLimitInterval.value = null
} }
return return
} }
@ -166,9 +166,9 @@ function updateRateLimitStatus() {
isRateLimited.value = false isRateLimited.value = false
rateLimitMessage.value = '' rateLimitMessage.value = ''
rateLimitExpires.value = 0 rateLimitExpires.value = 0
if (rateLimitInterval) { if (rateLimitInterval.value) {
clearInterval(rateLimitInterval) clearInterval(rateLimitInterval.value)
rateLimitInterval = null rateLimitInterval.value = null
} }
} else { } else {
// Still rate limited // Still rate limited
@ -177,8 +177,8 @@ function updateRateLimitStatus() {
rateLimitMessage.value = `Rate limited, available in ${secondsRemaining} second${secondsRemaining !== 1 ? 's' : ''}` rateLimitMessage.value = `Rate limited, available in ${secondsRemaining} second${secondsRemaining !== 1 ? 's' : ''}`
// Set up interval to update every second // Set up interval to update every second
if (!rateLimitInterval) { if (!rateLimitInterval.value) {
rateLimitInterval = setInterval(() => { rateLimitInterval.value = setInterval(() => {
updateRateLimitStatus() updateRateLimitStatus()
}, 1000) }, 1000)
} }
@ -218,10 +218,10 @@ async function pollExecutionUntilDone (trackingId) {
const pollTimeoutMs = 10 * 60 * 1000 const pollTimeoutMs = 10 * 60 * 1000
const deadline = Date.now() + pollTimeoutMs const deadline = Date.now() + pollTimeoutMs
while (Date.now() < deadline && isComponentMounted) { while (Date.now() < deadline && isComponentMounted.value) {
try { try {
const result = await window.client.executionStatus({ executionTrackingId: trackingId }) const result = await window.client.executionStatus({ executionTrackingId: trackingId })
if (!isComponentMounted) { if (!isComponentMounted.value) {
return return
} }
if (result.logEntry) { if (result.logEntry) {
@ -234,7 +234,7 @@ async function pollExecutionUntilDone (trackingId) {
console.error('Failed to poll execution status:', err) console.error('Failed to poll execution status:', err)
} }
if (!isComponentMounted) { if (!isComponentMounted.value) {
return return
} }
@ -287,17 +287,25 @@ async function startAction(actionArgs) {
function onLogEntryChanged(logEntry) { function onLogEntryChanged(logEntry) {
if (logEntry.executionFinished) { if (logEntry.executionFinished) {
onExecutionFinished(logEntry) onExecutionFinished(logEntry)
} else if (logEntry.queued && !logEntry.executionStarted) {
onExecutionQueued(logEntry)
} else { } else {
onExecutionStarted(logEntry) onExecutionStarted(logEntry)
} }
} }
function onExecutionQueued(_logEntry) {
isDisabled.value = true
updateDom('action-queued', '[Queued]')
}
function onExecutionStarted(logEntry) { function onExecutionStarted(logEntry) {
if (popupOnStart.value && popupOnStart.value.includes('execution-dialog')) { if (popupOnStart.value && popupOnStart.value.includes('execution-dialog')) {
router.push(`/logs/${logEntry.executionTrackingId}`) router.push(`/logs/${logEntry.executionTrackingId}`)
} }
isDisabled.value = true isDisabled.value = true
updateDom(null, title.value)
} }
function onExecutionFinished(logEntry) { function onExecutionFinished(logEntry) {
@ -358,10 +366,10 @@ onMounted(() => {
}) })
onUnmounted(() => { onUnmounted(() => {
isComponentMounted = false isComponentMounted.value = false
if (rateLimitInterval) { if (rateLimitInterval.value) {
clearInterval(rateLimitInterval) clearInterval(rateLimitInterval.value)
rateLimitInterval = null rateLimitInterval.value = null
} }
}) })
@ -449,6 +457,12 @@ defineExpose({
color: #721c24; color: #721c24;
} }
.action-button button.action-queued {
background: #e7f1ff !important;
border-color: #9ec5fe;
color: #084298;
}
.action-button button.action-nonzero-exit { .action-button button.action-nonzero-exit {
background: #f8d7da !important; background: #f8d7da !important;
border-color: #f5c6cb; border-color: #f5c6cb;

View File

@ -19,6 +19,10 @@ const statusText = computed(() => {
const logEntry = props.logEntry const logEntry = props.logEntry
if (!logEntry) return 'unknown' if (!logEntry) return 'unknown'
if (logEntry.queued && !logEntry.executionFinished) {
return 'Queued'
}
if (logEntry.executionFinished) { if (logEntry.executionFinished) {
if (logEntry.blocked) { if (logEntry.blocked) {
return 'Blocked' return 'Blocked'
@ -83,4 +87,4 @@ const statusClass = computed(() => {
} }
</style> </style>

View File

@ -1,5 +1,5 @@
<template> <template>
<button @click="navigateToDirectory" :class="component.cssClass"> <button class="directory-button" :class="component.cssClass" @click="navigateToDirectory">
<span class="icon" v-html="unicodeIcon"></span> <span class="icon" v-html="unicodeIcon"></span>
<span class="title">{{ component.title }}</span> <span class="title">{{ component.title }}</span>
</button> </button>
@ -45,11 +45,7 @@ function navigateToDirectory() {
<style> <style>
@layer components { @layer components {
.folder-container { .directory-button {
display: grid;
}
button {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
flex-grow: 1; flex-grow: 1;
@ -64,31 +60,31 @@ button {
font-size: .85em; font-size: .85em;
} }
button:hover { .directory-button:hover {
background-color: #f5f5f5; background-color: #f5f5f5;
border-color: #999; border-color: #999;
} }
button .icon { .directory-button .icon {
font-size: 3em; font-size: 3em;
flex-grow: 1; flex-grow: 1;
align-content: center; align-content: center;
} }
button .title { .directory-button .title {
font-weight: 500; font-weight: 500;
padding: 0.2em; padding: 0.2em;
} }
@media (prefers-color-scheme: dark) { @media (prefers-color-scheme: dark) {
button { .directory-button {
box-shadow: 0 0 .6em #000; box-shadow: 0 0 .6em #000;
background-color: #111; background-color: #111;
border-color: #000; border-color: #000;
color: #fff; color: #fff;
} }
button:hover { .directory-button:hover {
background-color: #222; background-color: #222;
border-color: #000; border-color: #000;
box-shadow: 0 0 6px #444; box-shadow: 0 0 6px #444;

View File

@ -49,6 +49,18 @@ const routes = [
] ]
} }
}, },
{
path: '/logs/queue',
name: 'LogsQueue',
component: () => import('./views/LogsQueueView.vue'),
meta: {
title: 'Execution Queue',
breadcrumb: [
{ name: "Logs", href: "/logs" },
{ name: "Queue" },
]
}
},
{ {
path: '/entities', path: '/entities',
name: 'Entities', name: 'Entities',

View File

@ -1,6 +1,9 @@
<template> <template>
<Section :title="t('logs.title')" :padding="false"> <Section :title="t('logs.title')" :padding="false">
<template #toolbar> <template #toolbar>
<router-link to="/logs/queue" class="button neutral">
{{ t('logs.queue') }}
</router-link>
<router-link to="/logs/calendar" class="button neutral"> <router-link to="/logs/calendar" class="button neutral">
{{ t('logs.calendar') }} {{ t('logs.calendar') }}
</router-link> </router-link>
@ -9,7 +12,15 @@
<path fill="currentColor" <path fill="currentColor"
d="m19.6 21l-6.3-6.3q-.75.6-1.725.95T9.5 16q-2.725 0-4.612-1.888T3 9.5t1.888-4.612T9.5 3t4.613 1.888T16 9.5q0 1.1-.35 2.075T14.7 13.3l6.3 6.3zM9.5 14q1.875 0 3.188-1.312T14 9.5t-1.312-3.187T9.5 5T6.313 6.313T5 9.5t1.313 3.188T9.5 14" /> d="m19.6 21l-6.3-6.3q-.75.6-1.725.95T9.5 16q-2.725 0-4.612-1.888T3 9.5t1.888-4.612T9.5 3t4.613 1.888T16 9.5q0 1.1-.35 2.075T14.7 13.3l6.3 6.3zM9.5 14q1.875 0 3.188-1.312T14 9.5t-1.312-3.187T9.5 5T6.313 6.313T5 9.5t1.313 3.188T9.5 14" />
</svg> </svg>
<input :placeholder="t('search-filter')" v-model="searchText" /> <input
:placeholder="t('logs.filter-placeholder')"
v-model="searchText"
list="logs-filter-suggestions"
:aria-invalid="filterError ? 'true' : 'false'"
/>
<datalist id="logs-filter-suggestions">
<option v-for="suggestion in filterSuggestions" :key="suggestion" :value="suggestion" />
</datalist>
<button :title="t('logs.clear-filter')" :disabled="!searchText" @click="clearSearch"> <button :title="t('logs.clear-filter')" :disabled="!searchText" @click="clearSearch">
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24"> <svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24">
<path fill="currentColor" <path fill="currentColor"
@ -19,8 +30,18 @@
</label> </label>
</template> </template>
<p class = "padding">{{ t('logs.page-description') }}</p> <div class="padding logs-intro">
<div v-show="filteredLogs.length > 0"> <p>{{ t('logs.page-description') }}</p>
<details class="filter-help">
<summary>{{ t('logs.filter-help-title') }}</summary>
<p>{{ t('logs.filter-help-intro') }}</p>
<p>{{ t('logs.filter-help-fields') }}</p>
<p><code>{{ t('logs.filter-help-examples') }}</code></p>
</details>
<p v-if="filterError" class="filter-error" role="alert">{{ filterError }}</p>
</div>
<div v-show="logs.length > 0">
<table class="logs-table"> <table class="logs-table">
<thead> <thead>
<tr> <tr>
@ -44,7 +65,7 @@
</tr> </tr>
</thead> </thead>
<tbody> <tbody>
<tr v-for="log in filteredLogs" :key="log.executionTrackingId" class="log-row" :title="log.actionTitle"> <tr v-for="log in logs" :key="log.executionTrackingId" class="log-row" :title="log.actionTitle">
<td class="timestamp">{{ formatTimestamp(log.datetimeStarted) }}</td> <td class="timestamp">{{ formatTimestamp(log.datetimeStarted) }}</td>
<td> <td>
<ActionIconGlyph class="icon" :glyph="log.actionIcon" /> <ActionIconGlyph class="icon" :glyph="log.actionIcon" />
@ -72,14 +93,21 @@
@page-size-change="handlePageSizeChange" itemTitle="execution logs" /> @page-size-change="handlePageSizeChange" itemTitle="execution logs" />
</div> </div>
<div v-show="selectedDate && filteredLogs.length === 0" class="empty-state"> <div v-show="logs.length === 0 && !loading && searchText && !filterError" class="empty-state padding">
<p>{{ t('logs.no-logs-for-filter') }}</p>
<button @click="clearSearch" class="button neutral">
{{ t('logs.clear-filter') }}
</button>
</div>
<div v-show="selectedDate && logs.length === 0 && !loading && !searchText" class="empty-state padding">
<p>{{ t('logs.no-logs-to-display') }} {{ formatDateFilter(selectedDate) }}.</p> <p>{{ t('logs.no-logs-to-display') }} {{ formatDateFilter(selectedDate) }}.</p>
<button @click="clearDateFilter" class="button neutral"> <button @click="clearDateFilter" class="button neutral">
{{ t('logs.clear-date-filter') }} {{ t('logs.clear-date-filter') }}
</button> </button>
</div> </div>
<div v-show="logs.length === 0 && !selectedDate" class="empty-state"> <div v-show="logs.length === 0 && !loading && !selectedDate && !searchText" class="empty-state padding">
<p>{{ t('logs.no-logs-to-display') }}</p> <p>{{ t('logs.no-logs-to-display') }}</p>
<router-link to="/">{{ t('return-to-index') }}</router-link> <router-link to="/">{{ t('return-to-index') }}</router-link>
</div> </div>
@ -87,8 +115,9 @@
</template> </template>
<script setup> <script setup>
import { ref, computed, onMounted, watch } from 'vue' import { ref, onMounted, watch } from 'vue'
import { useRoute, useRouter } from 'vue-router' import { useRoute, useRouter } from 'vue-router'
import { ConnectError, Code } from '@connectrpc/connect'
import Pagination from 'picocrank/vue/components/Pagination.vue' import Pagination from 'picocrank/vue/components/Pagination.vue'
import Section from 'picocrank/vue/components/Section.vue' import Section from 'picocrank/vue/components/Section.vue'
import { useI18n } from 'vue-i18n' import { useI18n } from 'vue-i18n'
@ -105,10 +134,20 @@ const currentPage = ref(1)
const loading = ref(false) const loading = ref(false)
const totalCount = ref(0) const totalCount = ref(0)
const selectedDate = ref(null) const selectedDate = ref(null)
const filterError = ref('')
let fetchTimer = null
const filterSuggestions = [
'!Update',
'Status != Completed',
'Status == Blocked',
'Status == Running',
'Action contains backup',
'User == guest'
]
const { t } = useI18n() const { t } = useI18n()
// Read date query parameter from route
function updateDateFromRoute() { function updateDateFromRoute() {
const dateParam = route.query.date const dateParam = route.query.date
if (dateParam) { if (dateParam) {
@ -116,75 +155,78 @@ function updateDateFromRoute() {
} else { } else {
selectedDate.value = null selectedDate.value = null
} }
// Re-fetch logs when date changes
fetchLogs() fetchLogs()
} }
// Watch for route changes to update date filter
watch(() => route.query.date, () => { watch(() => route.query.date, () => {
updateDateFromRoute() updateDateFromRoute()
}) })
const filteredLogs = computed(() => { watch(searchText, () => {
let result = logs.value currentPage.value = 1
scheduleFetchLogs()
// Date filtering is now done server-side, so we only need to filter by search text
if (searchText.value) {
const searchLower = searchText.value.toLowerCase()
result = result.filter(log =>
log.actionTitle.toLowerCase().includes(searchLower)
)
}
// Sort by timestamp with most recent first
return [...result].sort((a, b) => {
const dateA = a.datetimeStarted ? new Date(a.datetimeStarted).getTime() : 0
const dateB = b.datetimeStarted ? new Date(b.datetimeStarted).getTime() : 0
return dateB - dateA // Descending order (most recent first)
})
}) })
async function fetchLogs() { async function fetchLogs() {
loading.value = true loading.value = true
filterError.value = ''
try { try {
const startOffset = (currentPage.value - 1) * pageSize.value const startOffset = (currentPage.value - 1) * pageSize.value
const args = { const args = {
"startOffset": BigInt(startOffset), startOffset: BigInt(startOffset),
"pageSize": BigInt(pageSize.value), pageSize: BigInt(pageSize.value)
} }
// Add date filter if selected
if (selectedDate.value) { if (selectedDate.value) {
args.dateFilter = selectedDate.value args.dateFilter = selectedDate.value
} }
if (searchText.value.trim()) {
args.filter = searchText.value.trim()
}
const response = await window.client.getLogs(args) const response = await window.client.getLogs(args)
logs.value = response.logs logs.value = response.logs
totalCount.value = Number(response.totalCount) || 0 totalCount.value = Number(response.totalCount) || 0
} catch (err) { } catch (err) {
console.error('Failed to fetch logs:', err) console.error('Failed to fetch logs:', err)
if (err instanceof ConnectError && err.code === Code.InvalidArgument && searchText.value.trim()) {
filterError.value = `${t('logs.filter-error')} ${err.message}`
logs.value = []
totalCount.value = 0
return
}
window.showBigError('fetch-logs', 'getting logs', err, false) window.showBigError('fetch-logs', 'getting logs', err, false)
} finally { } finally {
loading.value = false loading.value = false
} }
} }
function scheduleFetchLogs() {
if (fetchTimer) {
clearTimeout(fetchTimer)
}
fetchTimer = setTimeout(() => {
fetchLogs()
}, 400)
}
function clearSearch() { function clearSearch() {
searchText.value = '' searchText.value = ''
currentPage.value = 1
fetchLogs()
} }
function clearDateFilter() { function clearDateFilter() {
selectedDate.value = null selectedDate.value = null
// Remove date query parameter from URL
const query = { ...route.query } const query = { ...route.query }
delete query.date delete query.date
router.push({ path: route.path, query }) router.push({ path: route.path, query })
} }
function formatDateFilter(dateString) { function formatDateFilter(dateString) {
// Format YYYY-MM-DD to a short format (e.g., "Jan 15, 2024")
try { try {
const date = new Date(dateString + 'T00:00:00') const date = new Date(dateString + 'T00:00:00')
return date.toLocaleDateString(undefined, { year: 'numeric', month: 'short', day: 'numeric' }) return date.toLocaleDateString(undefined, { year: 'numeric', month: 'short', day: 'numeric' })
@ -210,7 +252,7 @@ function handlePageChange(page) {
function handlePageSizeChange(newPageSize) { function handlePageSizeChange(newPageSize) {
pageSize.value = newPageSize pageSize.value = newPageSize
currentPage.value = 1 // Reset to first page currentPage.value = 1
fetchLogs() fetchLogs()
} }
@ -220,8 +262,29 @@ onMounted(() => {
</script> </script>
<style scoped> <style scoped>
.logs-view { .logs-intro {
padding: 1rem; display: flex;
flex-direction: column;
gap: 0.75rem;
}
.logs-intro p {
margin: 0;
}
.filter-help summary {
cursor: pointer;
font-weight: 600;
}
.filter-help code {
display: block;
white-space: pre-wrap;
margin-top: 0.5rem;
}
.filter-error {
color: var(--karma-bad-fg, #b00020);
} }
.input-with-icons { .input-with-icons {
@ -233,7 +296,7 @@ onMounted(() => {
border-radius: 0.25rem; border-radius: 0.25rem;
background: var(--section-background); background: var(--section-background);
width: 100%; width: 100%;
max-width: 300px; max-width: 360px;
} }
.input-with-icons input { .input-with-icons input {
@ -268,16 +331,6 @@ onMounted(() => {
font-size: 1.2em; font-size: 1.2em;
} }
.content {
color: #007bff;
text-decoration: none;
cursor: pointer;
}
.content:hover {
text-decoration: underline;
}
.annotation { .annotation {
font-weight: 500; font-weight: 500;
font-size: smaller; font-size: smaller;

View File

@ -0,0 +1,214 @@
<template>
<Section :title="t('logs.queue-title')" :padding="false">
<template #toolbar>
<router-link to="/logs" class="button neutral">
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24">
<path fill="currentColor" d="M20 11H7.83l5.59-5.59L12 4l-8 8l8 8l1.41-1.41L7.83 13H20z"/>
</svg>
{{ t('logs.back-to-list') }}
</router-link>
</template>
<p class="padding">{{ t('logs.queue-page-description') }}</p>
<div v-if="groups.length > 0" class="queue-groups padding">
<section v-for="group in groups" :key="group.bindingId" class="queue-group">
<header class="queue-group-header">
<ActionIconGlyph class="icon" :glyph="group.actionIcon" />
<div class="queue-group-title">
<h3>{{ group.actionTitle }}</h3>
<p v-if="group.entityPrefix" class="queue-entity">
{{ t('logs.queue-entity') }}: {{ group.entityPrefix }}
</p>
</div>
<span class="queue-group-limit annotation">
{{ t('logs.queue-group-active', { active: group.activeCount, max: group.maxConcurrent }) }}
</span>
</header>
<table class="logs-table">
<thead>
<tr>
<th>{{ t('logs.timestamp') }}</th>
<th>{{ t('logs.metadata') }}</th>
<th>{{ t('logs.status') }}</th>
</tr>
</thead>
<tbody>
<tr v-for="(entry, index) in group.entries" :key="entry.executionTrackingId" class="log-row">
<td class="timestamp">{{ formatTimestamp(entry.datetimeStarted) }}</td>
<td class="tags">
<span class="annotation">
<span class="annotation-key">User:</span>
<span class="annotation-val">{{ entry.user }}</span>
</span>
<span v-if="entry.tags && entry.tags.length > 0" class="tag-list">
<span v-for="tag in entry.tags" :key="tag" class="tag">{{ tag }}</span>
</span>
<span class="annotation">
<span class="annotation-key">ID:</span>
<router-link :to="`/logs/${entry.executionTrackingId}`">
{{ entry.executionTrackingId }}
</router-link>
</span>
</td>
<td class="exit-code">
<span class="annotation">
<span class="queue-position">{{ t('logs.queue-position', { position: index + 1 }) }}</span>
<span :class="queueStatusClass(entry)">{{ queueStatusText(entry) }}</span>
</span>
</td>
</tr>
</tbody>
</table>
</section>
</div>
<div v-else-if="!loading" class="empty-state padding">
<p>{{ t('logs.queue-empty') }}</p>
<router-link to="/logs">{{ t('logs.back-to-list') }}</router-link>
</div>
</Section>
</template>
<script setup>
import { ref, onMounted, onUnmounted } from 'vue'
import Section from 'picocrank/vue/components/Section.vue'
import ActionIconGlyph from '../components/ActionIconGlyph.vue'
import { useI18n } from 'vue-i18n'
const { t } = useI18n()
const groups = ref([])
const loading = ref(false)
function queueStatusText (entry) {
if (entry.executionStarted) {
return t('logs.queue-running')
}
return t('logs.queue-waiting')
}
function queueStatusClass (entry) {
return entry.executionStarted ? 'queue-status-running' : 'queue-status-waiting'
}
function formatTimestamp (timestamp) {
if (!timestamp) {
return 'Unknown'
}
try {
return new Date(timestamp).toLocaleString()
} catch (err) {
return timestamp
}
}
async function fetchQueue () {
loading.value = true
try {
const response = await window.client.getExecutionQueue({})
groups.value = response.groups || []
} catch (err) {
console.error('Failed to fetch execution queue:', err)
window.showBigError('fetch-queue', 'getting execution queue', err, false)
} finally {
loading.value = false
}
}
onMounted(() => {
fetchQueue()
window.addEventListener('EventExecutionStarted', fetchQueue)
window.addEventListener('EventExecutionFinished', fetchQueue)
})
onUnmounted(() => {
window.removeEventListener('EventExecutionStarted', fetchQueue)
window.removeEventListener('EventExecutionFinished', fetchQueue)
})
</script>
<style scoped>
.queue-groups {
display: flex;
flex-direction: column;
gap: 1.5rem;
}
.queue-group {
border: 1px solid var(--border-color, #ccc);
border-radius: 0.5rem;
overflow: hidden;
}
.queue-group-header {
display: flex;
align-items: center;
gap: 0.75rem;
padding: 0.75rem 1rem;
background: var(--section-background, #f8f9fa);
border-bottom: 1px solid var(--border-color, #ccc);
}
.queue-group-title {
flex: 1;
}
.queue-group-title h3 {
margin: 0;
font-size: 1rem;
}
.queue-entity {
margin: 0.25rem 0 0;
font-size: 0.85rem;
color: #666;
}
.queue-group-limit {
white-space: nowrap;
}
.icon {
font-size: 1.5em;
}
.timestamp {
font-family: monospace;
font-size: 0.875rem;
color: #666;
}
.annotation {
font-weight: 500;
font-size: smaller;
}
.queue-position {
margin-right: 0.5rem;
}
.queue-status-running {
color: var(--karma-warning-fg, #856404);
}
.queue-status-waiting {
color: #0d6efd;
}
.empty-state {
text-align: center;
padding: 2rem;
color: #666;
}
.empty-state a {
color: #007bff;
text-decoration: none;
}
.empty-state a:hover {
text-decoration: underline;
}
</style>

View File

@ -43,6 +43,15 @@
"logs.metadata": "Metadaten", "logs.metadata": "Metadaten",
"logs.no-logs-to-display": "Es gibt keine Protokolle zu anzeigen.", "logs.no-logs-to-display": "Es gibt keine Protokolle zu anzeigen.",
"logs.page-description": "Dies ist eine Liste von Protokollen von Aktionen, die ausgeführt wurden. Sie können die Liste nach Aktionstitel filtern.", "logs.page-description": "Dies ist eine Liste von Protokollen von Aktionen, die ausgeführt wurden. Sie können die Liste nach Aktionstitel filtern.",
"logs.queue": "Warteschlange",
"logs.queue-empty": "Derzeit gibt es keine aktiven oder wartenden Ausführungen.",
"logs.queue-entity": "Entität",
"logs.queue-group-active": "{active} aktiv (max. {max})",
"logs.queue-page-description": "Aktive und wartende Ausführungen, nach Aktion gruppiert. Einträge ohne Berechtigung werden ausgeblendet.",
"logs.queue-position": "#{position}",
"logs.queue-running": "Läuft",
"logs.queue-title": "Ausführungswarteschlange",
"logs.queue-waiting": "Wartend",
"logs.status": "Status", "logs.status": "Status",
"logs.timed-out": "Zeitüberschreitung", "logs.timed-out": "Zeitüberschreitung",
"logs.timestamp": "Zeitstempel", "logs.timestamp": "Zeitstempel",
@ -99,9 +108,25 @@
"logs.clear-filter": "Clear search filter", "logs.clear-filter": "Clear search filter",
"logs.completed": "Completed", "logs.completed": "Completed",
"logs.exit-code": "Exit code", "logs.exit-code": "Exit code",
"logs.filter-error": "Could not apply filter expression.",
"logs.filter-help-examples": "Examples: backup · !Update · Status != Completed · Status == Blocked · Action contains backup and Status == Completed",
"logs.filter-help-fields": "Fields: Status, Action, User, Output, Blocked, TimedOut, Running, ExitCode. Operators: ==, !=, contains. Prefix ! on a single word excludes matching entries.",
"logs.filter-help-intro": "Filters run on the server over entries you are allowed to view. Combine terms with and / or.",
"logs.filter-help-title": "Filter syntax",
"logs.filter-placeholder": "Filter logs (e.g. !Update or Status != Completed)",
"logs.metadata": "Metadata", "logs.metadata": "Metadata",
"logs.no-logs-for-filter": "No logs match the current filter.",
"logs.no-logs-to-display": "There are no logs to display.", "logs.no-logs-to-display": "There are no logs to display.",
"logs.page-description": "This is a list of logs from actions that have been executed. You can filter the list by action title.", "logs.page-description": "This is a list of logs from actions that have been executed. Use the filter box for search terms and expressions.",
"logs.queue": "Queue",
"logs.queue-empty": "There are no active or waiting executions right now.",
"logs.queue-entity": "Entity",
"logs.queue-group-active": "{active} active (max {max})",
"logs.queue-page-description": "Active and waiting executions grouped by action. Entries you are not permitted to view are hidden.",
"logs.queue-position": "#{position}",
"logs.queue-running": "Running",
"logs.queue-title": "Execution Queue",
"logs.queue-waiting": "Waiting",
"logs.status": "Status", "logs.status": "Status",
"logs.timed-out": "Timed out", "logs.timed-out": "Timed out",
"logs.timestamp": "Timestamp", "logs.timestamp": "Timestamp",
@ -161,6 +186,15 @@
"logs.metadata": "Metadatos", "logs.metadata": "Metadatos",
"logs.no-logs-to-display": "No hay registros para mostrar.", "logs.no-logs-to-display": "No hay registros para mostrar.",
"logs.page-description": "Esta es una lista de registros de acciones que han sido ejecutadas. Puede filtrar la lista por título de acción.", "logs.page-description": "Esta es una lista de registros de acciones que han sido ejecutadas. Puede filtrar la lista por título de acción.",
"logs.queue": "Cola",
"logs.queue-empty": "No hay ejecuciones activas o en espera en este momento.",
"logs.queue-entity": "Entidad",
"logs.queue-group-active": "{active} activas (máx. {max})",
"logs.queue-page-description": "Ejecuciones activas y en espera agrupadas por acción. Las entradas que no puede ver se ocultan.",
"logs.queue-position": "#{position}",
"logs.queue-running": "En ejecución",
"logs.queue-title": "Cola de ejecución",
"logs.queue-waiting": "En espera",
"logs.status": "Estado", "logs.status": "Estado",
"logs.timed-out": "Tiempo agotado", "logs.timed-out": "Tiempo agotado",
"logs.timestamp": "Marca de tiempo", "logs.timestamp": "Marca de tiempo",
@ -220,6 +254,15 @@
"logs.metadata": "Metadati", "logs.metadata": "Metadati",
"logs.no-logs-to-display": "Non ci sono registri da mostrare.", "logs.no-logs-to-display": "Non ci sono registri da mostrare.",
"logs.page-description": "Questa è una lista di registri delle azioni che sono state eseguite. Puoi filtrare la lista per titolo dell'azione.", "logs.page-description": "Questa è una lista di registri delle azioni che sono state eseguite. Puoi filtrare la lista per titolo dell'azione.",
"logs.queue": "Coda",
"logs.queue-empty": "Non ci sono esecuzioni attive o in attesa al momento.",
"logs.queue-entity": "Entità",
"logs.queue-group-active": "{active} attive (max {max})",
"logs.queue-page-description": "Esecuzioni attive e in attesa raggruppate per azione. Le voci non autorizzate sono nascoste.",
"logs.queue-position": "#{position}",
"logs.queue-running": "In esecuzione",
"logs.queue-title": "Coda di esecuzione",
"logs.queue-waiting": "In attesa",
"logs.status": "Stato", "logs.status": "Stato",
"logs.timed-out": "Tempo scaduto", "logs.timed-out": "Tempo scaduto",
"logs.timestamp": "Date e ora", "logs.timestamp": "Date e ora",
@ -279,6 +322,15 @@
"logs.metadata": "元数据", "logs.metadata": "元数据",
"logs.no-logs-to-display": "没有日志可显示。", "logs.no-logs-to-display": "没有日志可显示。",
"logs.page-description": "这是一个动作执行日志列表。您可以按动作标题过滤列表。", "logs.page-description": "这是一个动作执行日志列表。您可以按动作标题过滤列表。",
"logs.queue": "队列",
"logs.queue-empty": "当前没有正在运行或等待中的执行。",
"logs.queue-entity": "实体",
"logs.queue-group-active": "{active} 个活动(上限 {max})",
"logs.queue-page-description": "按动作分组显示正在运行和等待中的执行。您无权查看的条目会被隐藏。",
"logs.queue-position": "第 {position} 位",
"logs.queue-running": "运行中",
"logs.queue-title": "执行队列",
"logs.queue-waiting": "等待中",
"logs.status": "状态", "logs.status": "状态",
"logs.timed-out": "超时", "logs.timed-out": "超时",
"logs.timestamp": "时间戳", "logs.timestamp": "时间戳",

View File

@ -31,6 +31,15 @@ translations:
logs.calendar: Kalender logs.calendar: Kalender
logs.calendar-title: Protokoll-Kalender logs.calendar-title: Protokoll-Kalender
logs.back-to-list: Zurück zur Liste logs.back-to-list: Zurück zur Liste
logs.queue: Warteschlange
logs.queue-title: Ausführungswarteschlange
logs.queue-page-description: Aktive und wartende Ausführungen, nach Aktion gruppiert. Einträge ohne Berechtigung werden ausgeblendet.
logs.queue-empty: Derzeit gibt es keine aktiven oder wartenden Ausführungen.
logs.queue-group-active: "{active} aktiv (max. {max})"
logs.queue-waiting: Wartend
logs.queue-running: Läuft
logs.queue-position: "#{position}"
logs.queue-entity: Entität
diagnostics.get-support: Unterstützung erhalten diagnostics.get-support: Unterstützung erhalten
diagnostics.get-support-description: Wenn Sie Probleme mit OliveTin haben und eine Support-Anfrage stellen möchten, wäre es sehr hilfreich, einen sosreport von dieser Seite einzufügen. diagnostics.get-support-description: Wenn Sie Probleme mit OliveTin haben und eine Support-Anfrage stellen möchten, wäre es sehr hilfreich, einen sosreport von dieser Seite einzufügen.
diagnostics.where-to-find-help: Wo Sie Hilfe finden diagnostics.where-to-find-help: Wo Sie Hilfe finden

View File

@ -16,7 +16,14 @@ translations:
disconnected-banner-suffix-reconnecting: " since {disconnectedSince}. Trying reconnect…" disconnected-banner-suffix-reconnecting: " since {disconnectedSince}. Trying reconnect…"
login-button: Login login-button: Login
logs.title: Logs logs.title: Logs
logs.page-description: This is a list of logs from actions that have been executed. You can filter the list by action title. logs.page-description: This is a list of logs from actions that have been executed. Use the filter box for search terms and expressions.
logs.filter-placeholder: Filter logs (e.g. !Update or Status != Completed)
logs.filter-help-title: Filter syntax
logs.filter-help-intro: Filters run on the server over entries you are allowed to view. Combine terms with and / or.
logs.filter-help-fields: "Fields: Status, Action, User, Output, Blocked, TimedOut, Running, ExitCode. Operators: ==, !=, contains. Prefix ! on a single word excludes matching entries."
logs.filter-help-examples: "Examples: backup · !Update · Status != Completed · Status == Blocked · Action contains backup and Status == Completed"
logs.filter-error: Could not apply filter expression.
logs.no-logs-for-filter: No logs match the current filter.
logs.timestamp: Timestamp logs.timestamp: Timestamp
logs.action: Action logs.action: Action
logs.metadata: Metadata logs.metadata: Metadata
@ -31,6 +38,15 @@ translations:
logs.calendar: Calendar logs.calendar: Calendar
logs.calendar-title: Logs Calendar logs.calendar-title: Logs Calendar
logs.back-to-list: Back to List logs.back-to-list: Back to List
logs.queue: Queue
logs.queue-title: Execution Queue
logs.queue-page-description: Active and waiting executions grouped by action. Entries you are not permitted to view are hidden.
logs.queue-empty: There are no active or waiting executions right now.
logs.queue-group-active: "{active} active (max {max})"
logs.queue-waiting: Waiting
logs.queue-running: Running
logs.queue-position: "#{position}"
logs.queue-entity: Entity
diagnostics.get-support: Get support diagnostics.get-support: Get support
diagnostics.get-support-description: If you are having problems with OliveTin and want to raise a support request, it would be very helpful to include a sosreport from this page. diagnostics.get-support-description: If you are having problems with OliveTin and want to raise a support request, it would be very helpful to include a sosreport from this page.
diagnostics.where-to-find-help: Where to find help diagnostics.where-to-find-help: Where to find help

View File

@ -31,6 +31,15 @@ translations:
logs.calendar: Calendario logs.calendar: Calendario
logs.calendar-title: Calendario de Registros logs.calendar-title: Calendario de Registros
logs.back-to-list: Volver a la Lista logs.back-to-list: Volver a la Lista
logs.queue: Cola
logs.queue-title: Cola de ejecución
logs.queue-page-description: Ejecuciones activas y en espera agrupadas por acción. Las entradas que no puede ver se ocultan.
logs.queue-empty: No hay ejecuciones activas o en espera en este momento.
logs.queue-group-active: "{active} activas (máx. {max})"
logs.queue-waiting: En espera
logs.queue-running: En ejecución
logs.queue-position: "#{position}"
logs.queue-entity: Entidad
diagnostics.get-support: Obtener soporte diagnostics.get-support: Obtener soporte
diagnostics.get-support-description: Si tiene problemas con OliveTin y desea presentar una solicitud de soporte, sería muy útil incluir un sosreport de esta página. diagnostics.get-support-description: Si tiene problemas con OliveTin y desea presentar una solicitud de soporte, sería muy útil incluir un sosreport de esta página.
diagnostics.where-to-find-help: Dónde encontrar ayuda diagnostics.where-to-find-help: Dónde encontrar ayuda

View File

@ -31,6 +31,15 @@ translations:
logs.calendar: Calendario logs.calendar: Calendario
logs.calendar-title: Calendario dei Registri logs.calendar-title: Calendario dei Registri
logs.back-to-list: Torna all'Elenco logs.back-to-list: Torna all'Elenco
logs.queue: Coda
logs.queue-title: Coda di esecuzione
logs.queue-page-description: Esecuzioni attive e in attesa raggruppate per azione. Le voci non autorizzate sono nascoste.
logs.queue-empty: Non ci sono esecuzioni attive o in attesa al momento.
logs.queue-group-active: "{active} attive (max {max})"
logs.queue-waiting: In attesa
logs.queue-running: In esecuzione
logs.queue-position: "#{position}"
logs.queue-entity: Entità
diagnostics.get-support: Ottenere supporto diagnostics.get-support: Ottenere supporto
diagnostics.get-support-description: Se hai problemi con OliveTin e vuoi presentare una richiesta di supporto, sarebbe molto utile includere un sosreport da questa pagina. diagnostics.get-support-description: Se hai problemi con OliveTin e vuoi presentare una richiesta di supporto, sarebbe molto utile includere un sosreport da questa pagina.
diagnostics.where-to-find-help: Dove trovare aiuto diagnostics.where-to-find-help: Dove trovare aiuto

View File

@ -40,6 +40,15 @@ translations:
logs.calendar: 日历 logs.calendar: 日历
logs.calendar-title: 日志日历 logs.calendar-title: 日志日历
logs.back-to-list: 返回列表 logs.back-to-list: 返回列表
logs.queue: 队列
logs.queue-title: 执行队列
logs.queue-page-description: 按动作分组显示正在运行和等待中的执行。您无权查看的条目会被隐藏。
logs.queue-empty: 当前没有正在运行或等待中的执行。
logs.queue-group-active: "{active} 个活动(上限 {max})"
logs.queue-waiting: 等待中
logs.queue-running: 运行中
logs.queue-position: "第 {position} 位"
logs.queue-entity: 实体
diagnostics.get-support: 获取支持 diagnostics.get-support: 获取支持
diagnostics.get-support-description: 如果您在使用 OliveTin 时遇到问题并希望提交支持请求,从本页面包含 sosreport 将非常有帮助。 diagnostics.get-support-description: 如果您在使用 OliveTin 时遇到问题并希望提交支持请求,从本页面包含 sosreport 将非常有帮助。
diagnostics.where-to-find-help: 在哪里找到帮助 diagnostics.where-to-find-help: 在哪里找到帮助

View File

@ -136,6 +136,7 @@ message GetLogsRequest{
int64 start_offset = 1; int64 start_offset = 1;
string date_filter = 2; // Optional date filter in YYYY-MM-DD format string date_filter = 2; // Optional date filter in YYYY-MM-DD format
int64 page_size = 3; // Number of logs per page (optional; server default used if 0 or unset) int64 page_size = 3; // Number of logs per page (optional; server default used if 0 or unset)
string filter = 4; // Optional filter expression (see logs UI syntax help)
}; };
message LogEntry { message LogEntry {
@ -157,6 +158,8 @@ message LogEntry {
bool can_kill = 18; bool can_kill = 18;
string datetime_rate_limit_expires = 19; // Datetime when rate limit expires (empty string if not rate limited), format: "2006-01-02 15:04:05" string datetime_rate_limit_expires = 19; // Datetime when rate limit expires (empty string if not rate limited), format: "2006-01-02 15:04:05"
string binding_id = 20; // Binding ID for matching rate limits to action buttons string binding_id = 20; // Binding ID for matching rate limits to action buttons
bool queued = 21;
string queued_for_group = 22;
} }
message GetLogsResponse { message GetLogsResponse {
@ -180,6 +183,23 @@ message GetActionLogsResponse {
int64 start_offset = 5; int64 start_offset = 5;
} }
message GetExecutionQueueRequest {}
message ExecutionQueueGroup {
string binding_id = 1;
string action_title = 2;
string action_icon = 3;
int32 max_concurrent = 4;
int32 active_count = 5;
string entity_prefix = 6;
repeated LogEntry entries = 7;
}
message GetExecutionQueueResponse {
repeated ExecutionQueueGroup groups = 1;
int32 total_active = 2;
}
message ValidateArgumentTypeRequest { message ValidateArgumentTypeRequest {
string value = 1; string value = 1;
string type = 2; string type = 2;
@ -415,6 +435,8 @@ service OliveTinApiService {
rpc GetActionLogs(GetActionLogsRequest) returns (GetActionLogsResponse) {} rpc GetActionLogs(GetActionLogsRequest) returns (GetActionLogsResponse) {}
rpc GetExecutionQueue(GetExecutionQueueRequest) returns (GetExecutionQueueResponse) {}
rpc ValidateArgumentType(ValidateArgumentTypeRequest) returns (ValidateArgumentTypeResponse) {} rpc ValidateArgumentType(ValidateArgumentTypeRequest) returns (ValidateArgumentTypeResponse) {}
rpc WhoAmI(WhoAmIRequest) returns (WhoAmIResponse) {} rpc WhoAmI(WhoAmIRequest) returns (WhoAmIResponse) {}

View File

@ -63,6 +63,9 @@ const (
// OliveTinApiServiceGetActionLogsProcedure is the fully-qualified name of the OliveTinApiService's // OliveTinApiServiceGetActionLogsProcedure is the fully-qualified name of the OliveTinApiService's
// GetActionLogs RPC. // GetActionLogs RPC.
OliveTinApiServiceGetActionLogsProcedure = "/olivetin.api.v1.OliveTinApiService/GetActionLogs" OliveTinApiServiceGetActionLogsProcedure = "/olivetin.api.v1.OliveTinApiService/GetActionLogs"
// OliveTinApiServiceGetExecutionQueueProcedure is the fully-qualified name of the
// OliveTinApiService's GetExecutionQueue RPC.
OliveTinApiServiceGetExecutionQueueProcedure = "/olivetin.api.v1.OliveTinApiService/GetExecutionQueue"
// OliveTinApiServiceValidateArgumentTypeProcedure is the fully-qualified name of the // OliveTinApiServiceValidateArgumentTypeProcedure is the fully-qualified name of the
// OliveTinApiService's ValidateArgumentType RPC. // OliveTinApiService's ValidateArgumentType RPC.
OliveTinApiServiceValidateArgumentTypeProcedure = "/olivetin.api.v1.OliveTinApiService/ValidateArgumentType" OliveTinApiServiceValidateArgumentTypeProcedure = "/olivetin.api.v1.OliveTinApiService/ValidateArgumentType"
@ -121,6 +124,7 @@ type OliveTinApiServiceClient interface {
ExecutionStatus(context.Context, *connect.Request[v1.ExecutionStatusRequest]) (*connect.Response[v1.ExecutionStatusResponse], error) ExecutionStatus(context.Context, *connect.Request[v1.ExecutionStatusRequest]) (*connect.Response[v1.ExecutionStatusResponse], error)
GetLogs(context.Context, *connect.Request[v1.GetLogsRequest]) (*connect.Response[v1.GetLogsResponse], error) GetLogs(context.Context, *connect.Request[v1.GetLogsRequest]) (*connect.Response[v1.GetLogsResponse], error)
GetActionLogs(context.Context, *connect.Request[v1.GetActionLogsRequest]) (*connect.Response[v1.GetActionLogsResponse], error) GetActionLogs(context.Context, *connect.Request[v1.GetActionLogsRequest]) (*connect.Response[v1.GetActionLogsResponse], error)
GetExecutionQueue(context.Context, *connect.Request[v1.GetExecutionQueueRequest]) (*connect.Response[v1.GetExecutionQueueResponse], error)
ValidateArgumentType(context.Context, *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error) ValidateArgumentType(context.Context, *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error)
WhoAmI(context.Context, *connect.Request[v1.WhoAmIRequest]) (*connect.Response[v1.WhoAmIResponse], error) WhoAmI(context.Context, *connect.Request[v1.WhoAmIRequest]) (*connect.Response[v1.WhoAmIResponse], error)
SosReport(context.Context, *connect.Request[v1.SosReportRequest]) (*connect.Response[v1.SosReportResponse], error) SosReport(context.Context, *connect.Request[v1.SosReportRequest]) (*connect.Response[v1.SosReportResponse], error)
@ -209,6 +213,12 @@ func NewOliveTinApiServiceClient(httpClient connect.HTTPClient, baseURL string,
connect.WithSchema(oliveTinApiServiceMethods.ByName("GetActionLogs")), connect.WithSchema(oliveTinApiServiceMethods.ByName("GetActionLogs")),
connect.WithClientOptions(opts...), connect.WithClientOptions(opts...),
), ),
getExecutionQueue: connect.NewClient[v1.GetExecutionQueueRequest, v1.GetExecutionQueueResponse](
httpClient,
baseURL+OliveTinApiServiceGetExecutionQueueProcedure,
connect.WithSchema(oliveTinApiServiceMethods.ByName("GetExecutionQueue")),
connect.WithClientOptions(opts...),
),
validateArgumentType: connect.NewClient[v1.ValidateArgumentTypeRequest, v1.ValidateArgumentTypeResponse]( validateArgumentType: connect.NewClient[v1.ValidateArgumentTypeRequest, v1.ValidateArgumentTypeResponse](
httpClient, httpClient,
baseURL+OliveTinApiServiceValidateArgumentTypeProcedure, baseURL+OliveTinApiServiceValidateArgumentTypeProcedure,
@ -314,6 +324,7 @@ type oliveTinApiServiceClient struct {
executionStatus *connect.Client[v1.ExecutionStatusRequest, v1.ExecutionStatusResponse] executionStatus *connect.Client[v1.ExecutionStatusRequest, v1.ExecutionStatusResponse]
getLogs *connect.Client[v1.GetLogsRequest, v1.GetLogsResponse] getLogs *connect.Client[v1.GetLogsRequest, v1.GetLogsResponse]
getActionLogs *connect.Client[v1.GetActionLogsRequest, v1.GetActionLogsResponse] getActionLogs *connect.Client[v1.GetActionLogsRequest, v1.GetActionLogsResponse]
getExecutionQueue *connect.Client[v1.GetExecutionQueueRequest, v1.GetExecutionQueueResponse]
validateArgumentType *connect.Client[v1.ValidateArgumentTypeRequest, v1.ValidateArgumentTypeResponse] validateArgumentType *connect.Client[v1.ValidateArgumentTypeRequest, v1.ValidateArgumentTypeResponse]
whoAmI *connect.Client[v1.WhoAmIRequest, v1.WhoAmIResponse] whoAmI *connect.Client[v1.WhoAmIRequest, v1.WhoAmIResponse]
sosReport *connect.Client[v1.SosReportRequest, v1.SosReportResponse] sosReport *connect.Client[v1.SosReportRequest, v1.SosReportResponse]
@ -381,6 +392,11 @@ func (c *oliveTinApiServiceClient) GetActionLogs(ctx context.Context, req *conne
return c.getActionLogs.CallUnary(ctx, req) return c.getActionLogs.CallUnary(ctx, req)
} }
// GetExecutionQueue calls olivetin.api.v1.OliveTinApiService.GetExecutionQueue.
func (c *oliveTinApiServiceClient) GetExecutionQueue(ctx context.Context, req *connect.Request[v1.GetExecutionQueueRequest]) (*connect.Response[v1.GetExecutionQueueResponse], error) {
return c.getExecutionQueue.CallUnary(ctx, req)
}
// ValidateArgumentType calls olivetin.api.v1.OliveTinApiService.ValidateArgumentType. // ValidateArgumentType calls olivetin.api.v1.OliveTinApiService.ValidateArgumentType.
func (c *oliveTinApiServiceClient) ValidateArgumentType(ctx context.Context, req *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error) { func (c *oliveTinApiServiceClient) ValidateArgumentType(ctx context.Context, req *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error) {
return c.validateArgumentType.CallUnary(ctx, req) return c.validateArgumentType.CallUnary(ctx, req)
@ -468,6 +484,7 @@ type OliveTinApiServiceHandler interface {
ExecutionStatus(context.Context, *connect.Request[v1.ExecutionStatusRequest]) (*connect.Response[v1.ExecutionStatusResponse], error) ExecutionStatus(context.Context, *connect.Request[v1.ExecutionStatusRequest]) (*connect.Response[v1.ExecutionStatusResponse], error)
GetLogs(context.Context, *connect.Request[v1.GetLogsRequest]) (*connect.Response[v1.GetLogsResponse], error) GetLogs(context.Context, *connect.Request[v1.GetLogsRequest]) (*connect.Response[v1.GetLogsResponse], error)
GetActionLogs(context.Context, *connect.Request[v1.GetActionLogsRequest]) (*connect.Response[v1.GetActionLogsResponse], error) GetActionLogs(context.Context, *connect.Request[v1.GetActionLogsRequest]) (*connect.Response[v1.GetActionLogsResponse], error)
GetExecutionQueue(context.Context, *connect.Request[v1.GetExecutionQueueRequest]) (*connect.Response[v1.GetExecutionQueueResponse], error)
ValidateArgumentType(context.Context, *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error) ValidateArgumentType(context.Context, *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error)
WhoAmI(context.Context, *connect.Request[v1.WhoAmIRequest]) (*connect.Response[v1.WhoAmIResponse], error) WhoAmI(context.Context, *connect.Request[v1.WhoAmIRequest]) (*connect.Response[v1.WhoAmIResponse], error)
SosReport(context.Context, *connect.Request[v1.SosReportRequest]) (*connect.Response[v1.SosReportResponse], error) SosReport(context.Context, *connect.Request[v1.SosReportRequest]) (*connect.Response[v1.SosReportResponse], error)
@ -552,6 +569,12 @@ func NewOliveTinApiServiceHandler(svc OliveTinApiServiceHandler, opts ...connect
connect.WithSchema(oliveTinApiServiceMethods.ByName("GetActionLogs")), connect.WithSchema(oliveTinApiServiceMethods.ByName("GetActionLogs")),
connect.WithHandlerOptions(opts...), connect.WithHandlerOptions(opts...),
) )
oliveTinApiServiceGetExecutionQueueHandler := connect.NewUnaryHandler(
OliveTinApiServiceGetExecutionQueueProcedure,
svc.GetExecutionQueue,
connect.WithSchema(oliveTinApiServiceMethods.ByName("GetExecutionQueue")),
connect.WithHandlerOptions(opts...),
)
oliveTinApiServiceValidateArgumentTypeHandler := connect.NewUnaryHandler( oliveTinApiServiceValidateArgumentTypeHandler := connect.NewUnaryHandler(
OliveTinApiServiceValidateArgumentTypeProcedure, OliveTinApiServiceValidateArgumentTypeProcedure,
svc.ValidateArgumentType, svc.ValidateArgumentType,
@ -664,6 +687,8 @@ func NewOliveTinApiServiceHandler(svc OliveTinApiServiceHandler, opts ...connect
oliveTinApiServiceGetLogsHandler.ServeHTTP(w, r) oliveTinApiServiceGetLogsHandler.ServeHTTP(w, r)
case OliveTinApiServiceGetActionLogsProcedure: case OliveTinApiServiceGetActionLogsProcedure:
oliveTinApiServiceGetActionLogsHandler.ServeHTTP(w, r) oliveTinApiServiceGetActionLogsHandler.ServeHTTP(w, r)
case OliveTinApiServiceGetExecutionQueueProcedure:
oliveTinApiServiceGetExecutionQueueHandler.ServeHTTP(w, r)
case OliveTinApiServiceValidateArgumentTypeProcedure: case OliveTinApiServiceValidateArgumentTypeProcedure:
oliveTinApiServiceValidateArgumentTypeHandler.ServeHTTP(w, r) oliveTinApiServiceValidateArgumentTypeHandler.ServeHTTP(w, r)
case OliveTinApiServiceWhoAmIProcedure: case OliveTinApiServiceWhoAmIProcedure:
@ -743,6 +768,10 @@ func (UnimplementedOliveTinApiServiceHandler) GetActionLogs(context.Context, *co
return nil, connect.NewError(connect.CodeUnimplemented, errors.New("olivetin.api.v1.OliveTinApiService.GetActionLogs is not implemented")) return nil, connect.NewError(connect.CodeUnimplemented, errors.New("olivetin.api.v1.OliveTinApiService.GetActionLogs is not implemented"))
} }
func (UnimplementedOliveTinApiServiceHandler) GetExecutionQueue(context.Context, *connect.Request[v1.GetExecutionQueueRequest]) (*connect.Response[v1.GetExecutionQueueResponse], error) {
return nil, connect.NewError(connect.CodeUnimplemented, errors.New("olivetin.api.v1.OliveTinApiService.GetExecutionQueue is not implemented"))
}
func (UnimplementedOliveTinApiServiceHandler) ValidateArgumentType(context.Context, *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error) { func (UnimplementedOliveTinApiServiceHandler) ValidateArgumentType(context.Context, *connect.Request[v1.ValidateArgumentTypeRequest]) (*connect.Response[v1.ValidateArgumentTypeResponse], error) {
return nil, connect.NewError(connect.CodeUnimplemented, errors.New("olivetin.api.v1.OliveTinApiService.ValidateArgumentType is not implemented")) return nil, connect.NewError(connect.CodeUnimplemented, errors.New("olivetin.api.v1.OliveTinApiService.ValidateArgumentType is not implemented"))
} }

File diff suppressed because it is too large Load Diff

View File

@ -74,6 +74,7 @@ require (
github.com/docker/docker-credential-helpers v0.9.7 // indirect github.com/docker/docker-credential-helpers v0.9.7 // indirect
github.com/docker/go-connections v0.7.0 // indirect github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect github.com/docker/go-units v0.5.0 // indirect
github.com/expr-lang/expr v1.17.8 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-chi/chi/v5 v5.2.5 // indirect github.com/go-chi/chi/v5 v5.2.5 // indirect
github.com/go-logr/logr v1.4.3 // indirect github.com/go-logr/logr v1.4.3 // indirect

View File

@ -196,6 +196,8 @@ github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/expr-lang/expr v1.17.8 h1:W1loDTT+0PQf5YteHSTpju2qfUfNoBt4yw9+wOEU9VM=
github.com/expr-lang/expr v1.17.8/go.mod h1:8/vRC7+7HBzESEqt5kKpYXxrxkr31SaO8r40VO/1IT4=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=

View File

@ -385,6 +385,8 @@ func (api *oliveTinAPI) internalLogEntryToPb(logEntry *executor.InternalLogEntry
ExecutionFinished: logEntry.ExecutionFinished, ExecutionFinished: logEntry.ExecutionFinished,
User: logEntry.Username, User: logEntry.Username,
BindingId: logEntry.GetBindingId(), BindingId: logEntry.GetBindingId(),
Queued: logEntry.Queued,
QueuedForGroup: logEntry.QueuedForGroup,
DatetimeRateLimitExpires: calculateRateLimitExpires(api, logEntry), DatetimeRateLimitExpires: calculateRateLimitExpires(api, logEntry),
} }
@ -623,7 +625,10 @@ func (api *oliveTinAPI) GetLogs(ctx ctx.Context, req *connect.Request[apiv1.GetL
} }
pageSize := resolveLogsPageSize(req.Msg.GetPageSize(), api.cfg.LogHistoryPageSize) pageSize := resolveLogsPageSize(req.Msg.GetPageSize(), api.cfg.LogHistoryPageSize)
logEntries, paging := api.executor.GetLogTrackingIdsACL(api.cfg, user, req.Msg.StartOffset, pageSize, req.Msg.DateFilter) logEntries, paging, err := api.executor.GetLogTrackingIdsACL(api.cfg, user, req.Msg.StartOffset, pageSize, req.Msg.DateFilter, req.Msg.GetFilter())
if err != nil {
return nil, connect.NewError(connect.CodeInvalidArgument, err)
}
ret := &apiv1.GetLogsResponse{} ret := &apiv1.GetLogsResponse{}
for _, le := range logEntries { for _, le := range logEntries {
ret.Logs = append(ret.Logs, api.internalLogEntryToPb(le, user)) ret.Logs = append(ret.Logs, api.internalLogEntryToPb(le, user))

View File

@ -0,0 +1,76 @@
package api
import (
"context"
"testing"
"time"
"connectrpc.com/connect"
apiv1 "github.com/OliveTin/OliveTin/gen/olivetin/api/v1"
config "github.com/OliveTin/OliveTin/internal/config"
"github.com/OliveTin/OliveTin/internal/executor"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestGetLogsFilterExpression(t *testing.T) {
cfg := config.DefaultConfig()
cfg.Actions = []*config.Action{
{Title: "Update packages", Shell: "echo update"},
{Title: "Ping host", Shell: "echo ping"},
}
cfg.Sanitize()
ex := executor.DefaultExecutor(cfg)
ex.RebuildActionMap()
updateBinding := ex.FindBindingWithNoEntity(cfg.Actions[0])
pingBinding := ex.FindBindingWithNoEntity(cfg.Actions[1])
require.NotNil(t, updateBinding)
require.NotNil(t, pingBinding)
ex.SetLog(uuid.NewString(), finishedLogEntry(updateBinding, "Update packages", "Completed"))
ex.SetLog(uuid.NewString(), finishedLogEntry(pingBinding, "Ping host", "Blocked"))
ts, client := getNewTestServerAndClientWithExecutor(cfg, ex)
defer ts.Close()
resp, err := client.GetLogs(context.Background(), connect.NewRequest(&apiv1.GetLogsRequest{
Filter: "!Update",
}))
require.NoError(t, err)
require.Len(t, resp.Msg.Logs, 1)
assert.Equal(t, "Ping host", resp.Msg.Logs[0].ActionTitle)
}
func TestGetLogsInvalidFilterReturnsError(t *testing.T) {
cfg := config.DefaultConfig()
ts, client := getNewTestServerAndClient(cfg)
defer ts.Close()
_, err := client.GetLogs(context.Background(), connect.NewRequest(&apiv1.GetLogsRequest{
Filter: `SecretField == "x"`,
}))
require.Error(t, err)
assert.Equal(t, connect.CodeInvalidArgument, connect.CodeOf(err))
}
func finishedLogEntry(binding *executor.ActionBinding, title, status string) *executor.InternalLogEntry {
entry := &executor.InternalLogEntry{
Binding: binding,
DatetimeStarted: time.Now(),
DatetimeFinished: time.Now(),
ExecutionTrackingID: uuid.NewString(),
ActionTitle: title,
ExecutionFinished: true,
Username: "guest",
}
switch status {
case "Blocked":
entry.Blocked = true
case "Completed":
entry.ExitCode = 0
}
return entry
}

View File

@ -0,0 +1,85 @@
package api
import (
ctx "context"
"sort"
"connectrpc.com/connect"
apiv1 "github.com/OliveTin/OliveTin/gen/olivetin/api/v1"
"github.com/OliveTin/OliveTin/internal/auth"
authpublic "github.com/OliveTin/OliveTin/internal/auth/authpublic"
"github.com/OliveTin/OliveTin/internal/executor"
)
func (api *oliveTinAPI) GetExecutionQueue(ctx ctx.Context, req *connect.Request[apiv1.GetExecutionQueueRequest]) (*connect.Response[apiv1.GetExecutionQueueResponse], error) {
user := auth.UserFromApiCall(ctx, req, api.cfg)
if err := api.checkDashboardAccess(user); err != nil {
return nil, err
}
active := api.executor.GetActiveExecutionsACL(api.cfg, user)
groups := buildExecutionQueueGroups(active, user, api)
return connect.NewResponse(&apiv1.GetExecutionQueueResponse{
Groups: groups,
TotalActive: int32(len(active)),
}), nil
}
func buildExecutionQueueGroups(active []*executor.InternalLogEntry, user *authpublic.AuthenticatedUser, api *oliveTinAPI) []*apiv1.ExecutionQueueGroup {
grouped := make(map[string]*apiv1.ExecutionQueueGroup)
for _, entry := range active {
bindingID := entry.GetBindingId()
group := grouped[bindingID]
if group == nil {
group = newExecutionQueueGroup(entry)
grouped[bindingID] = group
}
group.Entries = append(group.Entries, api.internalLogEntryToPb(entry, user))
}
groups := make([]*apiv1.ExecutionQueueGroup, 0, len(grouped))
for _, group := range grouped {
sortQueueEntries(group.Entries)
group.ActiveCount = int32(len(group.Entries))
groups = append(groups, group)
}
sortExecutionQueueGroups(groups)
return groups
}
func newExecutionQueueGroup(entry *executor.InternalLogEntry) *apiv1.ExecutionQueueGroup {
group := &apiv1.ExecutionQueueGroup{
BindingId: entry.GetBindingId(),
ActionTitle: entry.ActionTitle,
ActionIcon: entry.ActionIcon,
EntityPrefix: entry.EntityPrefix,
}
if entry.Binding != nil && entry.Binding.Action != nil {
group.MaxConcurrent = int32(entry.Binding.Action.MaxConcurrent)
}
return group
}
func sortQueueEntries(entries []*apiv1.LogEntry) {
sort.Slice(entries, func(i, j int) bool {
return entries[i].DatetimeStarted < entries[j].DatetimeStarted
})
}
func sortExecutionQueueGroups(groups []*apiv1.ExecutionQueueGroup) {
sort.Slice(groups, func(i, j int) bool {
left := groups[i].ActionTitle
right := groups[j].ActionTitle
if left == right {
return groups[i].EntityPrefix < groups[j].EntityPrefix
}
return left < right
})
}

View File

@ -0,0 +1,86 @@
package api
import (
"context"
"testing"
"time"
"connectrpc.com/connect"
apiv1 "github.com/OliveTin/OliveTin/gen/olivetin/api/v1"
config "github.com/OliveTin/OliveTin/internal/config"
"github.com/OliveTin/OliveTin/internal/executor"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestGetExecutionQueueGroupsByBinding(t *testing.T) {
cfg := config.DefaultConfig()
cfg.Actions = []*config.Action{
{Title: "backup", Shell: "sleep 1", MaxConcurrent: 1},
{Title: "ping", Shell: "echo ping"},
}
cfg.Sanitize()
ex := executor.DefaultExecutor(cfg)
ex.RebuildActionMap()
backupBinding := ex.FindBindingWithNoEntity(cfg.Actions[0])
pingBinding := ex.FindBindingWithNoEntity(cfg.Actions[1])
require.NotNil(t, backupBinding)
require.NotNil(t, pingBinding)
backupRunning := newAPIQueueLogEntry(backupBinding, true, false)
backupWaiting := newAPIQueueLogEntry(backupBinding, false, false)
pingRunning := newAPIQueueLogEntry(pingBinding, true, false)
ex.SetLog(backupRunning.ExecutionTrackingID, backupRunning)
ex.SetLog(backupWaiting.ExecutionTrackingID, backupWaiting)
ex.SetLog(pingRunning.ExecutionTrackingID, pingRunning)
ts, client := getNewTestServerAndClientWithExecutor(cfg, ex)
defer ts.Close()
resp, err := client.GetExecutionQueue(context.Background(), connect.NewRequest(&apiv1.GetExecutionQueueRequest{}))
require.NoError(t, err)
require.NotNil(t, resp)
assert.Equal(t, int32(3), resp.Msg.TotalActive)
require.Len(t, resp.Msg.Groups, 2)
var backupGroup *apiv1.ExecutionQueueGroup
for _, group := range resp.Msg.Groups {
if group.BindingId == backupBinding.ID {
backupGroup = group
}
}
require.NotNil(t, backupGroup)
assert.Equal(t, "backup", backupGroup.ActionTitle)
assert.Equal(t, int32(1), backupGroup.MaxConcurrent)
assert.Equal(t, int32(2), backupGroup.ActiveCount)
require.Len(t, backupGroup.Entries, 2)
assert.False(t, backupGroup.Entries[1].ExecutionStarted)
assert.True(t, backupGroup.Entries[0].ExecutionStarted)
}
func newAPIQueueLogEntry(binding *executor.ActionBinding, started bool, finished bool) *executor.InternalLogEntry {
startedAt := time.Now().Add(-time.Minute)
if started {
startedAt = time.Now().Add(-2 * time.Minute)
}
entry := &executor.InternalLogEntry{
Binding: binding,
DatetimeStarted: startedAt,
ExecutionTrackingID: uuid.NewString(),
ActionTitle: binding.Action.Title,
ExecutionStarted: started,
ExecutionFinished: finished,
}
if finished {
entry.DatetimeFinished = time.Now()
}
return entry
}

View File

@ -25,7 +25,10 @@ import (
func getNewTestServerAndClient(injectedConfig *config.Config) (*httptest.Server, apiv1connect.OliveTinApiServiceClient) { func getNewTestServerAndClient(injectedConfig *config.Config) (*httptest.Server, apiv1connect.OliveTinApiServiceClient) {
ex := executor.DefaultExecutor(injectedConfig) ex := executor.DefaultExecutor(injectedConfig)
ex.RebuildActionMap() ex.RebuildActionMap()
return getNewTestServerAndClientWithExecutor(injectedConfig, ex)
}
func getNewTestServerAndClientWithExecutor(injectedConfig *config.Config, ex *executor.Executor) (*httptest.Server, apiv1connect.OliveTinApiServiceClient) {
apiPath, apiHandler := GetNewHandler(ex) apiPath, apiHandler := GetNewHandler(ex)
mux := http.NewServeMux() mux := http.NewServeMux()

View File

@ -33,6 +33,13 @@ type Action struct {
PopupOnStart string `koanf:"popupOnStart"` PopupOnStart string `koanf:"popupOnStart"`
SaveLogs SaveLogsConfig `koanf:"saveLogs"` SaveLogs SaveLogsConfig `koanf:"saveLogs"`
EnabledExpression string `koanf:"enabledExpression"` EnabledExpression string `koanf:"enabledExpression"`
Groups []string `koanf:"groups"`
}
// ActionGroup defines shared limits and metadata for a set of actions.
type ActionGroup struct {
MaxConcurrent int `koanf:"maxConcurrent"`
Icon string `koanf:"icon"`
} }
// ActionArgument objects appear on Actions. // ActionArgument objects appear on Actions.
@ -134,6 +141,7 @@ type Config struct {
LogLevel string `koanf:"logLevel"` LogLevel string `koanf:"logLevel"`
LogDebugOptions LogDebugOptions `koanf:"logDebugOptions"` LogDebugOptions LogDebugOptions `koanf:"logDebugOptions"`
LogHistoryPageSize int64 `koanf:"logHistoryPageSize"` LogHistoryPageSize int64 `koanf:"logHistoryPageSize"`
ActionGroups map[string]*ActionGroup `koanf:"actionGroups"`
Actions []*Action `koanf:"actions"` Actions []*Action `koanf:"actions"`
Entities []*EntityFile `koanf:"entities"` Entities []*EntityFile `koanf:"entities"`
Dashboards []*DashboardComponent `koanf:"dashboards"` Dashboards []*DashboardComponent `koanf:"dashboards"`

View File

@ -2,7 +2,6 @@ package config
import ( import (
"fmt" "fmt"
"runtime"
"strings" "strings"
"text/template" "text/template"
@ -19,7 +18,6 @@ func (cfg *Config) Sanitize() {
cfg.sanitizeLogHistoryPageSize() cfg.sanitizeLogHistoryPageSize()
cfg.sanitizeLocalUsers() cfg.sanitizeLocalUsers()
cfg.sanitizeSecurityHeaders() cfg.sanitizeSecurityHeaders()
cfg.sanitizeServiceLogs()
// log.Infof("cfg %p", cfg) // log.Infof("cfg %p", cfg)
@ -29,6 +27,8 @@ func (cfg *Config) Sanitize() {
cfg.sanitizeDashboardsForInlineActions() cfg.sanitizeDashboardsForInlineActions()
cfg.sanitizeActionGroupReferences()
if err := cfg.validateReservedActionArgumentNames(); err != nil { if err := cfg.validateReservedActionArgumentNames(); err != nil {
log.Fatalf("%v", err) log.Fatalf("%v", err)
} }
@ -170,16 +170,6 @@ func (cfg *Config) sanitizeLogLevel() {
} }
} }
func (cfg *Config) sanitizeServiceLogs() {
if cfg.ServiceLogs.Directory == "" {
return
}
if runtime.GOOS != "windows" {
log.Errorf("serviceLogs.directory is configured but this option is only supported on Windows")
}
}
func (action *Action) sanitize(cfg *Config) { func (action *Action) sanitize(cfg *Config) {
if action.Timeout < 3 { if action.Timeout < 3 {
action.Timeout = 3 action.Timeout = 3
@ -193,11 +183,64 @@ func (action *Action) sanitize(cfg *Config) {
action.MaxConcurrent = 1 action.MaxConcurrent = 1
} }
action.Groups = dedupeStrings(action.Groups)
for idx := range action.Arguments { for idx := range action.Arguments {
action.Arguments[idx].sanitize() action.Arguments[idx].sanitize()
} }
} }
func dedupeStrings(values []string) []string {
seen := make(map[string]struct{}, len(values))
out := make([]string, 0, len(values))
for _, value := range values {
out = appendUniqueString(out, seen, value)
}
return out
}
func appendUniqueString(out []string, seen map[string]struct{}, value string) []string {
if value == "" {
return out
}
if _, found := seen[value]; found {
return out
}
seen[value] = struct{}{}
return append(out, value)
}
func (cfg *Config) sanitizeActionGroupReferences() {
for _, action := range cfg.Actions {
for _, groupName := range action.Groups {
cfg.warnInvalidActionGroupReference(action, groupName)
}
}
}
func (cfg *Config) warnInvalidActionGroupReference(action *Action, groupName string) {
group, found := cfg.ActionGroups[groupName]
if !found {
log.WithFields(log.Fields{
"actionTitle": action.Title,
"groupName": groupName,
}).Warn("Action references unknown action group")
return
}
if group == nil || group.MaxConcurrent < 1 {
log.WithFields(log.Fields{
"actionTitle": action.Title,
"groupName": groupName,
}).Warn("Action references action group that will not be enforced at runtime")
}
}
func (cfg *Config) sanitizeAuthRequireGuestsToLogin() { func (cfg *Config) sanitizeAuthRequireGuestsToLogin() {
if cfg.AuthRequireGuestsToLogin { if cfg.AuthRequireGuestsToLogin {
log.Infof("AuthRequireGuestsToLogin is enabled. All defaultPermissions will be set to false") log.Infof("AuthRequireGuestsToLogin is enabled. All defaultPermissions will be set to false")

View File

@ -1,11 +1,8 @@
package config package config
import ( import (
"bytes"
"runtime"
"testing" "testing"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -110,6 +107,24 @@ func TestValidateReservedActionArgumentNames(t *testing.T) {
assert.Contains(t, err.Error(), `action "Reserved arg" argument "ot_custom" uses reserved prefix "ot_"`) assert.Contains(t, err.Error(), `action "Reserved arg" argument "ot_custom" uses reserved prefix "ot_"`)
} }
func TestSanitizeActionGroupsDedupesGroupNames(t *testing.T) {
c := DefaultConfig()
c.ActionGroups = map[string]*ActionGroup{
"unity": {MaxConcurrent: 1},
}
c.Actions = append(c.Actions, &Action{
Title: "Build",
Shell: "true",
Groups: []string{"unity", "unity", ""},
})
c.Sanitize()
action := c.findAction("Build")
require.NotNil(t, action)
assert.Equal(t, []string{"unity"}, action.Groups)
}
func TestValidateReservedActionArgumentNamesAllowsNonReserved(t *testing.T) { func TestValidateReservedActionArgumentNamesAllowsNonReserved(t *testing.T) {
c := DefaultConfig() c := DefaultConfig()
c.Actions = append(c.Actions, &Action{ c.Actions = append(c.Actions, &Action{
@ -163,22 +178,3 @@ func TestValidateUniqueLocalUserAPIKeys(t *testing.T) {
}) })
require.NoError(t, err) require.NoError(t, err)
} }
func TestSanitizeServiceLogsUnsupportedPlatform(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("serviceLogs.directory platform check only applies on non-Windows")
}
var logBuffer bytes.Buffer
previousOutput := logrus.StandardLogger().Out
logrus.SetOutput(&logBuffer)
t.Cleanup(func() {
logrus.SetOutput(previousOutput)
})
cfg := DefaultConfig()
cfg.ServiceLogs.Directory = "/var/log/OliveTin"
cfg.Sanitize()
assert.Contains(t, logBuffer.String(), "serviceLogs.directory is configured but this option is only supported on Windows")
}

View File

@ -6,6 +6,7 @@ import (
authpublic "github.com/OliveTin/OliveTin/internal/auth/authpublic" authpublic "github.com/OliveTin/OliveTin/internal/auth/authpublic"
config "github.com/OliveTin/OliveTin/internal/config" config "github.com/OliveTin/OliveTin/internal/config"
"github.com/OliveTin/OliveTin/internal/entities" "github.com/OliveTin/OliveTin/internal/entities"
"github.com/OliveTin/OliveTin/internal/logfilter"
"github.com/OliveTin/OliveTin/internal/tpl" "github.com/OliveTin/OliveTin/internal/tpl"
"github.com/google/uuid" "github.com/google/uuid"
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
@ -71,6 +72,9 @@ type Executor struct {
listeners []listener listeners []listener
chainOfCommand []executorStepFunc chainOfCommand []executorStepFunc
groupQueue []*queuedExecution
groupQueueMu sync.Mutex
} }
// ExecutionRequest is a request to execute an action. It's passed to an // ExecutionRequest is a request to execute an action. It's passed to an
@ -84,11 +88,54 @@ type ExecutionRequest struct {
AuthenticatedUser *authpublic.AuthenticatedUser AuthenticatedUser *authpublic.AuthenticatedUser
TriggerDepth int TriggerDepth int
logEntry *InternalLogEntry logEntry *InternalLogEntry
finalParsedCommand string finalParsedCommand string
execArgs []string execArgs []string
useDirectExec bool useDirectExec bool
executor *Executor executor *Executor
skipRequestRegistration bool
}
func (req *ExecutionRequest) mutateLogEntry(mutator func(*InternalLogEntry)) {
if req.executor == nil {
mutator(req.logEntry)
return
}
req.executor.logmutex.Lock()
defer req.executor.logmutex.Unlock()
mutator(req.logEntry)
}
// LogEntrySnapshot is a copy of selected log entry fields for race-safe reads.
type LogEntrySnapshot struct {
Queued bool
Blocked bool
ExecutionStarted bool
ExecutionFinished bool
ExitCode int32
Output string
}
// SnapshotLog returns a copy of selected log entry fields under read lock.
func (e *Executor) SnapshotLog(trackingID string) (LogEntrySnapshot, bool) {
e.logmutex.RLock()
defer e.logmutex.RUnlock()
entry, found := e.logs[trackingID]
if !found {
return LogEntrySnapshot{}, false
}
return LogEntrySnapshot{
Queued: entry.Queued,
Blocked: entry.Blocked,
ExecutionStarted: entry.ExecutionStarted,
ExecutionFinished: entry.ExecutionFinished,
ExitCode: entry.ExitCode,
Output: entry.Output,
}, true
} }
// InternalLogEntry objects are created by an Executor, and represent the final // InternalLogEntry objects are created by an Executor, and represent the final
@ -101,6 +148,8 @@ type InternalLogEntry struct {
Output string Output string
TimedOut bool TimedOut bool
Blocked bool Blocked bool
Queued bool
QueuedForGroup string
ExitCode int32 ExitCode int32
Tags []string Tags []string
ExecutionStarted bool ExecutionStarted bool
@ -338,9 +387,22 @@ func paginateFilteredLogs(filtered []*InternalLogEntry, startOffset int64, pageC
// GetLogTrackingIdsACL returns logs filtered by ACL visibility for the user and // GetLogTrackingIdsACL returns logs filtered by ACL visibility for the user and
// paginated correctly based on the filtered set. // paginated correctly based on the filtered set.
// dateFilter is optional and should be in YYYY-MM-DD format. If empty, no date filtering is applied. // dateFilter is optional and should be in YYYY-MM-DD format. If empty, no date filtering is applied.
func (e *Executor) GetLogTrackingIdsACL(cfg *config.Config, user *authpublic.AuthenticatedUser, startOffset int64, pageCount int64, dateFilter string) ([]*InternalLogEntry, *PagingResult) { // expressionFilter is an optional filter expression applied after ACL checks.
func (e *Executor) GetLogTrackingIdsACL(cfg *config.Config, user *authpublic.AuthenticatedUser, startOffset int64, pageCount int64, dateFilter string, expressionFilter string) ([]*InternalLogEntry, *PagingResult, error) {
filtered := e.filterLogsByACL(cfg, user, dateFilter) filtered := e.filterLogsByACL(cfg, user, dateFilter)
return paginateFilteredLogs(filtered, startOffset, pageCount)
program, err := logfilter.Compile(expressionFilter)
if err != nil {
return nil, nil, err
}
filtered, err = applyLogFilter(filtered, program)
if err != nil {
return nil, nil, err
}
logs, paging := paginateFilteredLogs(filtered, startOffset, pageCount)
return logs, paging, nil
} }
func (e *Executor) GetLog(trackingID string) (*InternalLogEntry, bool) { func (e *Executor) GetLog(trackingID string) (*InternalLogEntry, bool) {
@ -369,7 +431,7 @@ func (e *Executor) GetLogsByBindingId(bindingId string) []*InternalLogEntry {
// shouldCountExecution checks if a log entry should be counted for rate limiting. // shouldCountExecution checks if a log entry should be counted for rate limiting.
func shouldCountExecution(logEntry *InternalLogEntry, windowStart time.Time) bool { func shouldCountExecution(logEntry *InternalLogEntry, windowStart time.Time) bool {
return !logEntry.Blocked && logEntry.DatetimeStarted.After(windowStart) return !logEntry.Blocked && !logEntry.Queued && logEntry.DatetimeStarted.After(windowStart)
} }
// updateOldestExecution updates the oldest execution time if this entry is older. // updateOldestExecution updates the oldest execution time if this entry is older.
@ -483,19 +545,45 @@ func (e *Executor) GetTimeUntilAvailable(binding *ActionBinding) int64 {
return maxExpiryTime.Unix() return maxExpiryTime.Unix()
} }
func (e *Executor) SetLog(trackingID string, entry *InternalLogEntry) { func (e *Executor) SetLog(trackingID string, entry *InternalLogEntry) string {
e.logmutex.Lock() e.logmutex.Lock()
defer e.logmutex.Unlock()
if _, found := e.logs[trackingID]; found || !isValidTrackingID(trackingID) {
trackingID = uuid.NewString()
entry.ExecutionTrackingID = trackingID
}
entry.Index = int64(len(e.logsTrackingIdsByDate)) entry.Index = int64(len(e.logsTrackingIdsByDate))
e.logs[trackingID] = entry e.logs[trackingID] = entry
e.logsTrackingIdsByDate = append(e.logsTrackingIdsByDate, trackingID) e.logsTrackingIdsByDate = append(e.logsTrackingIdsByDate, trackingID)
e.logmutex.Unlock() return trackingID
} }
// ExecRequest processes an ExecutionRequest // ExecRequest processes an ExecutionRequest
func (e *Executor) ExecRequest(req *ExecutionRequest) (*sync.WaitGroup, string) { func (e *Executor) ExecRequest(req *ExecutionRequest) (*sync.WaitGroup, string) {
e.initializeExecRequest(req)
log.Tracef("executor.ExecRequest(): trackingID=%s bindingID=%s", req.TrackingID, bindingIDForTrace(req))
req.TrackingID = e.SetLog(req.TrackingID, req.logEntry)
wg := new(sync.WaitGroup)
wg.Add(1)
go func() {
queued := e.execChain(req, wg)
if !queued {
wg.Done()
}
}()
return wg, req.TrackingID
}
func (e *Executor) initializeExecRequest(req *ExecutionRequest) {
if req.AuthenticatedUser == nil { if req.AuthenticatedUser == nil {
req.AuthenticatedUser = auth.UserGuest(req.Cfg) req.AuthenticatedUser = auth.UserGuest(req.Cfg)
} }
@ -513,56 +601,84 @@ func (e *Executor) ExecRequest(req *ExecutionRequest) (*sync.WaitGroup, string)
ActionIcon: "&#x1f4a9;", ActionIcon: "&#x1f4a9;",
Username: req.AuthenticatedUser.Username, Username: req.AuthenticatedUser.Username,
} }
_, isDuplicate := e.GetLog(req.TrackingID)
if isDuplicate || !isValidTrackingID(req.TrackingID) {
req.TrackingID = uuid.NewString()
}
// Update the log entry with the final tracking ID
req.logEntry.ExecutionTrackingID = req.TrackingID
log.Tracef("executor.ExecRequest(): %v", req)
e.SetLog(req.TrackingID, req.logEntry)
wg := new(sync.WaitGroup)
wg.Add(1)
go func() {
e.execChain(req)
defer wg.Done()
}()
return wg, req.TrackingID
} }
func (e *Executor) execChain(req *ExecutionRequest) { func bindingIDForTrace(req *ExecutionRequest) string {
for _, step := range e.chainOfCommand { if req.Binding == nil {
return ""
}
return req.Binding.ID
}
func (e *Executor) execChain(req *ExecutionRequest, wg *sync.WaitGroup) bool {
if !req.skipRequestRegistration {
finished, queued := e.registerOrQueueRequest(req, wg)
if finished || queued {
return queued
}
}
e.runExecutionSteps(req)
e.finishExecChain(req)
return false
}
func (e *Executor) registerOrQueueRequest(req *ExecutionRequest, wg *sync.WaitGroup) (finished bool, queued bool) {
if !stepRequestAction(req) {
e.finishExecChain(req)
return true, false
}
if !actionNeedsGroupLimit(req) || e.groupsHaveCapacityForActive(req) {
return false, false
}
return e.queueRequestAfterACL(req, wg)
}
func (e *Executor) queueRequestAfterACL(req *ExecutionRequest, wg *sync.WaitGroup) (finished bool, queued bool) {
if !stepACLCheck(req) {
e.finishExecChain(req)
return true, false
}
e.queueRequest(req, wg)
notifyListenersStarted(req)
return false, true
}
func (e *Executor) runExecutionSteps(req *ExecutionRequest) {
for _, step := range e.chainOfCommand[1:] {
if !step(req) { if !step(req) {
break break
} }
} }
}
// Ensure DatetimeFinished is set even if execution was blocked early func (e *Executor) finishExecChain(req *ExecutionRequest) {
if req.logEntry.DatetimeFinished.IsZero() { req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.DatetimeFinished = time.Now() if entry.DatetimeFinished.IsZero() {
} entry.DatetimeFinished = time.Now()
}
req.logEntry.ExecutionFinished = true entry.ExecutionFinished = true
})
// This isn't a step, because we want to notify all listeners, irrespective
// of how many steps were actually executed.
notifyListenersFinished(req) notifyListenersFinished(req)
e.drainGroupQueue()
} }
func getConcurrentCount(req *ExecutionRequest) int { func getConcurrentCount(req *ExecutionRequest) int {
concurrentCount := 0 concurrentCount := 0
req.executor.logmutex.RLock() req.executor.logmutex.RLock()
logs := req.executor.LogsByBindingId[req.Binding.ID]
for _, log := range req.executor.GetLogsByBindingId(req.Binding.ID) { for _, logEntry := range logs {
if !log.ExecutionFinished { if !logEntry.ExecutionFinished && !logEntry.Queued {
concurrentCount += 1 concurrentCount += 1
} }
} }
@ -583,8 +699,10 @@ func stepConcurrencyCheck(req *ExecutionRequest) bool {
"maxConcurrent": req.Binding.Action.MaxConcurrent, "maxConcurrent": req.Binding.Action.MaxConcurrent,
}).Warnf("Blocked from executing due to concurrency limit") }).Warnf("Blocked from executing due to concurrency limit")
req.logEntry.Output = "Blocked from executing due to concurrency limit" req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.Blocked = true entry.Output = "Blocked from executing due to concurrency limit"
entry.Blocked = true
})
return false return false
} }
@ -603,24 +721,35 @@ func parseDuration(rate config.RateSpec) time.Duration {
return duration return duration
} }
//gocyclo:ignore func entityPrefixForRequest(req *ExecutionRequest) string {
func getExecutionsCount(rate config.RateSpec, req *ExecutionRequest) int {
executions := -1 // Because we will find ourself when checking execution logs
duration := parseDuration(rate)
then := time.Now().Add(-duration)
currentEntityPrefix := ""
if req.Binding != nil && req.Binding.Entity != nil { if req.Binding != nil && req.Binding.Entity != nil {
currentEntityPrefix = req.Binding.Entity.UniqueKey return req.Binding.Entity.UniqueKey
} }
for _, logEntry := range req.executor.GetLogsByBindingId(req.Binding.ID) {
if logEntry.EntityPrefix != currentEntityPrefix {
continue
}
if logEntry.DatetimeStarted.After(then) && !logEntry.Blocked {
return ""
}
func rateExecutionMatchesScope(logEntry *InternalLogEntry, req *ExecutionRequest, entityPrefix string) bool {
if logEntry.EntityPrefix != entityPrefix {
return false
}
return !logEntry.Queued && logEntry.ExecutionTrackingID != req.TrackingID
}
func logEntryStartedInWindow(logEntry *InternalLogEntry, windowStart time.Time) bool {
return logEntry.DatetimeStarted.After(windowStart) && !logEntry.Blocked
}
func rateExecutionCountsForRate(logEntry *InternalLogEntry, req *ExecutionRequest, entityPrefix string, windowStart time.Time) bool {
return rateExecutionMatchesScope(logEntry, req, entityPrefix) && logEntryStartedInWindow(logEntry, windowStart)
}
func countRateExecutions(logs []*InternalLogEntry, req *ExecutionRequest, entityPrefix string, windowStart time.Time) int {
executions := 0
for _, logEntry := range logs {
if rateExecutionCountsForRate(logEntry, req, entityPrefix, windowStart) {
executions += 1 executions += 1
} }
} }
@ -628,6 +757,18 @@ func getExecutionsCount(rate config.RateSpec, req *ExecutionRequest) int {
return executions return executions
} }
func getExecutionsCount(rate config.RateSpec, req *ExecutionRequest) int {
duration := parseDuration(rate)
then := time.Now().Add(-duration)
req.executor.logmutex.RLock()
logs := req.executor.LogsByBindingId[req.Binding.ID]
executions := countRateExecutions(logs, req, entityPrefixForRequest(req), then)
req.executor.logmutex.RUnlock()
return executions
}
func stepRateCheck(req *ExecutionRequest) bool { func stepRateCheck(req *ExecutionRequest) bool {
for _, rate := range req.Binding.Action.MaxRate { for _, rate := range req.Binding.Action.MaxRate {
executions := getExecutionsCount(rate, req) executions := getExecutionsCount(rate, req)
@ -640,8 +781,10 @@ func stepRateCheck(req *ExecutionRequest) bool {
"duration": rate.Duration, "duration": rate.Duration,
}).Infof("Blocked from executing due to rate limit") }).Infof("Blocked from executing due to rate limit")
req.logEntry.Output = "Blocked from executing due to rate limit" req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.Blocked = true entry.Output = "Blocked from executing due to rate limit"
entry.Blocked = true
})
return false return false
} }
} }
@ -653,8 +796,10 @@ func stepACLCheck(req *ExecutionRequest) bool {
canExec := acl.IsAllowedExec(req.Cfg, req.AuthenticatedUser, req.Binding.Action) canExec := acl.IsAllowedExec(req.Cfg, req.AuthenticatedUser, req.Binding.Action)
if !canExec { if !canExec {
req.logEntry.Output = "ACL check failed. Blocked from executing." req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.Blocked = true entry.Output = "ACL check failed. Blocked from executing."
entry.Blocked = true
})
log.WithFields(log.Fields{ log.WithFields(log.Fields{
"actionTitle": req.logEntry.ActionTitle, "actionTitle": req.logEntry.ActionTitle,
@ -792,7 +937,9 @@ func hasExec(req *ExecutionRequest) bool {
} }
func fail(req *ExecutionRequest, err error) bool { func fail(req *ExecutionRequest, err error) bool {
req.logEntry.Output = err.Error() req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output = err.Error()
})
log.Warn(err.Error()) log.Warn(err.Error())
return false return false
} }
@ -826,14 +973,16 @@ func stepRequestActionHasBinding(req *ExecutionRequest) bool {
} }
func stepRequestActionPopulateLogEntry(req *ExecutionRequest) { func stepRequestActionPopulateLogEntry(req *ExecutionRequest) {
req.logEntry.Binding = req.Binding req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.ActionConfigTitle = req.Binding.Action.Title entry.Binding = req.Binding
req.logEntry.ActionTitle = tpl.ParseTemplateOfActionBeforeExec(req.Binding.Action.Title, req.Binding.Entity) entry.ActionConfigTitle = req.Binding.Action.Title
req.logEntry.ActionIcon = req.Binding.Action.Icon entry.ActionTitle = tpl.ParseTemplateOfActionBeforeExec(req.Binding.Action.Title, req.Binding.Entity)
req.logEntry.Tags = req.Tags entry.ActionIcon = req.Binding.Action.Icon
if req.Binding.Entity != nil { entry.Tags = req.Tags
req.logEntry.EntityPrefix = req.Binding.Entity.UniqueKey if req.Binding.Entity != nil {
} entry.EntityPrefix = req.Binding.Entity.UniqueKey
}
})
} }
func stepRequestActionRegisterLog(req *ExecutionRequest) { func stepRequestActionRegisterLog(req *ExecutionRequest) {
@ -856,7 +1005,9 @@ func stepLogStart(req *ExecutionRequest) bool {
} }
func stepLogFinish(req *ExecutionRequest) bool { func stepLogFinish(req *ExecutionRequest) bool {
req.logEntry.ExecutionFinished = true req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.ExecutionFinished = true
})
log.WithFields(log.Fields{ log.WithFields(log.Fields{
"actionTitle": req.logEntry.ActionTitle, "actionTitle": req.logEntry.ActionTitle,
@ -880,10 +1031,14 @@ func notifyListenersStarted(req *ExecutionRequest) {
} }
} }
func appendErrorToStderr(err error, logEntry *InternalLogEntry) { func appendErrorToStderr(req *ExecutionRequest, err error) {
if err != nil { if err == nil {
logEntry.Output = err.Error() + "\n\n" + logEntry.Output return
} }
req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output = err.Error() + "\n\n" + entry.Output
})
} }
type OutputStreamer struct { type OutputStreamer struct {
@ -926,31 +1081,41 @@ func stepExec(req *ExecutionRequest) bool {
streamer := &OutputStreamer{Req: req} streamer := &OutputStreamer{Req: req}
cmd := buildCommand(ctx, req) cmd := buildCommand(ctx, req)
if cmd == nil { if cmd == nil {
req.logEntry.Output = "Cannot execute: no command arguments provided" req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output = "Cannot execute: no command arguments provided"
})
log.Warn("Cannot execute: no command arguments provided") log.Warn("Cannot execute: no command arguments provided")
return false return false
} }
prepareCommand(cmd, streamer, req) prepareCommand(cmd, streamer, req)
runerr := cmd.Start() runerr := cmd.Start()
req.logEntry.Process = cmd.Process req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Process = cmd.Process
})
ctx.setProcess(cmd.Process) ctx.setProcess(cmd.Process)
waiterr := cmd.Wait() waiterr := cmd.Wait()
req.logEntry.ExitCode = int32(cmd.ProcessState.ExitCode()) req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.Output = streamer.String() entry.ExitCode = int32(cmd.ProcessState.ExitCode())
entry.Output = streamer.String()
})
appendErrorToStderr(runerr, req.logEntry) appendErrorToStderr(req, runerr)
appendErrorToStderr(waiterr, req.logEntry) appendErrorToStderr(req, waiterr)
if ctx.Err() == context.DeadlineExceeded { if ctx.Err() == context.DeadlineExceeded {
log.WithFields(log.Fields{ log.WithFields(log.Fields{
"actionTitle": req.logEntry.ActionTitle, "actionTitle": req.logEntry.ActionTitle,
}).Warnf("Action timed out") }).Warnf("Action timed out")
req.logEntry.TimedOut = true req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.Output += "OliveTin::timeout - this action timed out after " + fmt.Sprintf("%v", req.Binding.Action.Timeout) + " seconds. If you need more time for this action, set a longer timeout. See https://docs.olivetin.app/action_customization/timeouts.html for more help." entry.TimedOut = true
entry.Output += "OliveTin::timeout - this action timed out after " + fmt.Sprintf("%v", req.Binding.Action.Timeout) + " seconds. If you need more time for this action, set a longer timeout. See https://docs.olivetin.app/action_customization/timeouts.html for more help."
})
} }
req.logEntry.DatetimeFinished = time.Now() req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.DatetimeFinished = time.Now()
})
return true return true
} }
@ -966,7 +1131,9 @@ func prepareCommand(cmd *exec.Cmd, streamer *OutputStreamer, req *ExecutionReque
cmd.Stdout = streamer cmd.Stdout = streamer
cmd.Stderr = streamer cmd.Stderr = streamer
cmd.Env = buildEnv(req.Arguments) cmd.Env = buildEnv(req.Arguments)
req.logEntry.ExecutionStarted = true req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.ExecutionStarted = true
})
} }
func stepExecAfter(req *ExecutionRequest) bool { func stepExecAfter(req *ExecutionRequest) bool {
@ -991,24 +1158,28 @@ func stepExecAfter(req *ExecutionRequest) bool {
waiterr := cmd.Wait() waiterr := cmd.Wait()
req.logEntry.Output += "\n" req.mutateLogEntry(func(entry *InternalLogEntry) {
req.logEntry.Output += "OliveTin::shellAfterCompleted stdout\n" entry.Output += "\n"
req.logEntry.Output += stdout.String() entry.Output += "OliveTin::shellAfterCompleted stdout\n"
entry.Output += stdout.String()
entry.Output += "OliveTin::shellAfterCompleted stderr\n"
entry.Output += stderr.String()
entry.Output += "OliveTin::shellAfterCompleted errors and summary\n"
})
req.logEntry.Output += "OliveTin::shellAfterCompleted stderr\n" appendErrorToStderr(req, runerr)
req.logEntry.Output += stderr.String() appendErrorToStderr(req, waiterr)
req.logEntry.Output += "OliveTin::shellAfterCompleted errors and summary\n"
appendErrorToStderr(runerr, req.logEntry)
appendErrorToStderr(waiterr, req.logEntry)
if ctx.Err() == context.DeadlineExceeded { if ctx.Err() == context.DeadlineExceeded {
req.logEntry.Output += "Your shellAfterCompleted command timed out." req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output += "Your shellAfterCompleted command timed out."
})
} }
req.logEntry.Output += fmt.Sprintf("Your shellAfterCompleted exited with code %v\n", cmd.ProcessState.ExitCode()) req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output += fmt.Sprintf("Your shellAfterCompleted exited with code %v\n", cmd.ProcessState.ExitCode())
req.logEntry.Output += "OliveTin::shellAfterCompleted output complete\n" entry.Output += "OliveTin::shellAfterCompleted output complete\n"
})
return true return true
} }
@ -1026,7 +1197,9 @@ func buildShellAfterCommand(ctx context.Context, req *ExecutionRequest, stdout,
finalParsedCommand, err := tpl.ParseTemplateWithActionContext(req.Binding.Action.ShellAfterCompleted, req.Binding.Entity, args) finalParsedCommand, err := tpl.ParseTemplateWithActionContext(req.Binding.Action.ShellAfterCompleted, req.Binding.Entity, args)
if err != nil { if err != nil {
msg := "Could not prepare shellAfterCompleted command: " + err.Error() + "\n" msg := "Could not prepare shellAfterCompleted command: " + err.Error() + "\n"
req.logEntry.Output += msg req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output += msg
})
log.Warn(msg) log.Warn(msg)
return nil, nil, nil return nil, nil, nil
} }
@ -1061,7 +1234,9 @@ func stepTrigger(req *ExecutionRequest) bool {
"actionTitle": req.logEntry.ActionTitle, "actionTitle": req.logEntry.ActionTitle,
"depth": req.TriggerDepth, "depth": req.TriggerDepth,
}).Warnf("Trigger action reached maximum depth of %v. Not triggering further actions.", MaxTriggerDepth) }).Warnf("Trigger action reached maximum depth of %v. Not triggering further actions.", MaxTriggerDepth)
req.logEntry.Output += fmt.Sprintf("OliveTin::trigger - this action reached maximum trigger depth of %v. Not triggering further actions.", MaxTriggerDepth) req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Output += fmt.Sprintf("OliveTin::trigger - this action reached maximum trigger depth of %v. Not triggering further actions.", MaxTriggerDepth)
})
return true return true
} }

View File

@ -0,0 +1,203 @@
package executor
import (
"fmt"
"slices"
"sync"
config "github.com/OliveTin/OliveTin/internal/config"
log "github.com/sirupsen/logrus"
)
type groupLimit struct {
name string
maxConcurrent int
}
type queuedExecution struct {
req *ExecutionRequest
wg *sync.WaitGroup
}
func actionGroupLimits(req *ExecutionRequest) []groupLimit {
if !hasActionGroupContext(req) {
return nil
}
limits := make([]groupLimit, 0, len(req.Binding.Action.Groups))
for _, groupName := range req.Binding.Action.Groups {
if limit, ok := groupLimitFromConfig(req.Cfg, groupName); ok {
limits = append(limits, limit)
}
}
return limits
}
func hasActionGroupContext(req *ExecutionRequest) bool {
return req != nil && req.Binding != nil && req.Binding.Action != nil && req.Cfg != nil
}
func groupLimitFromConfig(cfg *config.Config, groupName string) (groupLimit, bool) {
group, found := cfg.ActionGroups[groupName]
if !found || group == nil || group.MaxConcurrent < 1 {
return groupLimit{}, false
}
return groupLimit{name: groupName, maxConcurrent: group.MaxConcurrent}, true
}
func actionNeedsGroupLimit(req *ExecutionRequest) bool {
return len(actionGroupLimits(req)) > 0
}
func actionInGroup(action *config.Action, groupName string) bool {
if action == nil {
return false
}
return slices.Contains(action.Groups, groupName)
}
func (e *Executor) countActiveInGroup(groupName string) int {
e.logmutex.RLock()
defer e.logmutex.RUnlock()
return e.countActiveInGroupLocked(groupName)
}
func (e *Executor) countActiveInGroupLocked(groupName string) int {
count := 0
for _, logEntry := range e.logs {
if logEntryIsActiveInGroup(logEntry, groupName) {
count++
}
}
return count
}
func logEntryIsActiveInGroup(logEntry *InternalLogEntry, groupName string) bool {
if inactiveLogEntry(logEntry) {
return false
}
return actionInGroup(logEntry.Binding.Action, groupName)
}
func inactiveLogEntry(logEntry *InternalLogEntry) bool {
if logEntry == nil {
return true
}
return logEntryIsInactive(logEntry)
}
func logEntryIsInactive(logEntry *InternalLogEntry) bool {
if logEntry.ExecutionFinished || logEntry.Queued {
return true
}
return logEntry.Binding == nil || logEntry.Binding.Action == nil
}
func (e *Executor) groupsHaveCapacityForActive(req *ExecutionRequest) bool {
for _, limit := range actionGroupLimits(req) {
if e.countActiveInGroup(limit.name) >= (limit.maxConcurrent + 1) {
return false
}
}
return true
}
func (e *Executor) groupsHaveCapacityForQueued(req *ExecutionRequest) bool {
for _, limit := range actionGroupLimits(req) {
if e.countActiveInGroup(limit.name) >= limit.maxConcurrent {
return false
}
}
return true
}
func firstFullGroupName(e *Executor, req *ExecutionRequest) string {
for _, limit := range actionGroupLimits(req) {
if e.countActiveInGroup(limit.name) >= (limit.maxConcurrent + 1) {
return limit.name
}
}
return ""
}
func firstFullGroupNameLocked(e *Executor, req *ExecutionRequest) string {
for _, limit := range actionGroupLimits(req) {
if e.countActiveInGroupLocked(limit.name) >= (limit.maxConcurrent + 1) {
return limit.name
}
}
return ""
}
func (e *Executor) queueRequest(req *ExecutionRequest, wg *sync.WaitGroup) {
e.groupQueueMu.Lock()
var groupName string
req.mutateLogEntry(func(entry *InternalLogEntry) {
groupName = firstFullGroupNameLocked(e, req)
entry.Queued = true
entry.QueuedForGroup = groupName
entry.Output = fmt.Sprintf("Queued waiting for action group %q", groupName)
})
e.groupQueue = append(e.groupQueue, &queuedExecution{req: req, wg: wg})
e.groupQueueMu.Unlock()
e.drainGroupQueue()
log.WithFields(log.Fields{
"actionTitle": req.logEntry.ActionTitle,
"groupName": groupName,
}).Infof("Action queued due to action group concurrency limit")
}
func (e *Executor) drainGroupQueue() {
e.groupQueueMu.Lock()
if len(e.groupQueue) == 0 {
e.groupQueueMu.Unlock()
return
}
next := e.groupQueue[0]
if !e.groupsHaveCapacityForQueued(next.req) {
e.groupQueueMu.Unlock()
return
}
e.groupQueue = e.groupQueue[1:]
next.req.mutateLogEntry(func(entry *InternalLogEntry) {
entry.Queued = false
entry.QueuedForGroup = ""
})
e.groupQueueMu.Unlock()
go e.runDequeuedExecution(next)
}
func (e *Executor) runDequeuedExecution(queued *queuedExecution) {
req := queued.req
req.skipRequestRegistration = true
e.runExecutionSteps(req)
e.finishExecChain(req)
queued.wg.Done()
}

View File

@ -0,0 +1,277 @@
package executor
import (
"sync"
"testing"
"time"
"github.com/OliveTin/OliveTin/internal/auth"
config "github.com/OliveTin/OliveTin/internal/config"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func testGroupExecutor(actions []*config.Action, groups map[string]*config.ActionGroup) (*Executor, *config.Config) {
cfg := config.DefaultConfig()
cfg.ActionGroups = groups
cfg.Actions = actions
cfg.Sanitize()
e := DefaultExecutor(cfg)
e.RebuildActionMap()
return e, cfg
}
func TestGroupConcurrencyQueuesSecondAction(t *testing.T) {
t.Parallel()
slowAction := &config.Action{
Title: "Unity Job 1",
Shell: "sleep 2",
Groups: []string{"unity"},
}
fastAction := &config.Action{
Title: "Unity Job 2",
Shell: "echo queued-run",
Groups: []string{"unity"},
}
e, cfg := testGroupExecutor(
[]*config.Action{slowAction, fastAction},
map[string]*config.ActionGroup{
"unity": {MaxConcurrent: 1},
},
)
binding1 := e.FindBindingWithNoEntity(slowAction)
binding2 := e.FindBindingWithNoEntity(fastAction)
require.NotNil(t, binding1)
require.NotNil(t, binding2)
wg1, tracking1 := e.ExecRequest(&ExecutionRequest{
Binding: binding1,
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
waitUntilExecutionStarted(t, e, tracking1)
wg2, tracking2 := e.ExecRequest(&ExecutionRequest{
Binding: binding2,
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
require.Eventually(t, func() bool {
snapshot, ok := e.SnapshotLog(tracking2)
return ok && snapshot.Queued
}, time.Second, 10*time.Millisecond)
wg1.Wait()
wg2.Wait()
snapshot, ok := e.SnapshotLog(tracking2)
require.True(t, ok)
assert.False(t, snapshot.Queued)
assert.False(t, snapshot.Blocked)
assert.Equal(t, int32(0), snapshot.ExitCode)
assert.Contains(t, snapshot.Output, "queued-run")
}
func TestDifferentGroupsRunConcurrently(t *testing.T) {
t.Parallel()
actionA := &config.Action{
Title: "Group A Job",
Shell: "sleep 1",
Groups: []string{"groupA"},
}
actionB := &config.Action{
Title: "Group B Job",
Shell: "echo group-b",
Groups: []string{"groupB"},
}
e, cfg := testGroupExecutor(
[]*config.Action{actionA, actionB},
map[string]*config.ActionGroup{
"groupA": {MaxConcurrent: 1},
"groupB": {MaxConcurrent: 1},
},
)
wg1, tracking1 := e.ExecRequest(&ExecutionRequest{
Binding: e.FindBindingWithNoEntity(actionA),
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
waitUntilExecutionStarted(t, e, tracking1)
wg2, tracking2 := e.ExecRequest(&ExecutionRequest{
Binding: e.FindBindingWithNoEntity(actionB),
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
require.Eventually(t, func() bool {
snapshot, ok := e.SnapshotLog(tracking2)
return ok && snapshot.ExecutionFinished && !snapshot.Queued
}, 2*time.Second, 20*time.Millisecond)
wg1.Wait()
wg2.Wait()
snapshot, ok := e.SnapshotLog(tracking2)
require.True(t, ok)
assert.Contains(t, snapshot.Output, "group-b")
}
func TestPerActionConcurrencyStillBlocksWithoutQueue(t *testing.T) {
t.Parallel()
action := &config.Action{
Title: "Single binding",
Shell: "sleep 1",
MaxConcurrent: 1,
}
e, cfg := testGroupExecutor([]*config.Action{action}, nil)
binding := e.FindBindingWithNoEntity(action)
wg1, tracking1 := e.ExecRequest(&ExecutionRequest{
Binding: binding,
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
waitUntilExecutionStarted(t, e, tracking1)
wg2, tracking2 := e.ExecRequest(&ExecutionRequest{
Binding: binding,
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
wg1.Wait()
wg2.Wait()
snapshot, ok := e.SnapshotLog(tracking2)
require.True(t, ok)
assert.True(t, snapshot.Blocked)
assert.False(t, snapshot.Queued)
}
func waitUntilExecutionStarted(t *testing.T, e *Executor, trackingID string) {
t.Helper()
require.Eventually(t, func() bool {
snapshot, ok := e.SnapshotLog(trackingID)
return ok && snapshot.ExecutionStarted
}, 2*time.Second, 10*time.Millisecond)
}
func assertWaitGroupPending(t *testing.T, wg *sync.WaitGroup) {
t.Helper()
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
t.Fatal("wait group completed before queued execution finished")
case <-time.After(100 * time.Millisecond):
}
}
func assertWaitGroupCompletes(t *testing.T, wg *sync.WaitGroup) {
t.Helper()
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("wait group did not complete after queue drained")
}
}
func TestStartActionAndWaitWaitsForQueuedExecution(t *testing.T) {
t.Parallel()
first := &config.Action{
Title: "Hold group",
Shell: "sleep 1",
Groups: []string{"unity"},
}
second := &config.Action{
Title: "Wait in queue",
Shell: "echo waited",
Groups: []string{"unity"},
}
e, cfg := testGroupExecutor(
[]*config.Action{first, second},
map[string]*config.ActionGroup{
"unity": {MaxConcurrent: 1},
},
)
wg1, tracking1 := e.ExecRequest(&ExecutionRequest{
Binding: e.FindBindingWithNoEntity(first),
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
waitUntilExecutionStarted(t, e, tracking1)
wg2, tracking2 := e.ExecRequest(&ExecutionRequest{
Binding: e.FindBindingWithNoEntity(second),
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
assertWaitGroupPending(t, wg2)
wg1.Wait()
assertWaitGroupCompletes(t, wg2)
snapshot, ok := e.SnapshotLog(tracking2)
require.True(t, ok)
assert.Contains(t, snapshot.Output, "waited")
}
func TestUnknownActionGroupReferenceWarnsAndSkipsLimit(t *testing.T) {
t.Parallel()
action := &config.Action{
Title: "Unknown group action",
Shell: "echo ok",
Groups: []string{"missing"},
}
e, cfg := testGroupExecutor([]*config.Action{action}, map[string]*config.ActionGroup{})
wg, tracking := e.ExecRequest(&ExecutionRequest{
Binding: e.FindBindingWithNoEntity(action),
Cfg: cfg,
AuthenticatedUser: auth.UserFromSystem(cfg, "testuser"),
})
wg.Wait()
snapshot, ok := e.SnapshotLog(tracking)
require.True(t, ok)
assert.False(t, snapshot.Queued)
assert.Equal(t, int32(0), snapshot.ExitCode)
}

View File

@ -0,0 +1,51 @@
package executor
import (
"fmt"
"github.com/OliveTin/OliveTin/internal/logfilter"
"github.com/expr-lang/expr/vm"
)
func filterRecordFromEntry(entry *InternalLogEntry) logfilter.Record {
return logfilter.Record{
Status: logfilter.StatusLabel(entry.ExecutionFinished, entry.Blocked, entry.TimedOut, entry.Queued),
Action: entry.ActionTitle,
User: entry.Username,
Tags: entry.Tags,
Blocked: entry.Blocked,
TimedOut: entry.TimedOut,
Running: !entry.ExecutionFinished,
ExitCode: entry.ExitCode,
Output: entry.Output,
}
}
func applyLogFilter(entries []*InternalLogEntry, program *vm.Program) ([]*InternalLogEntry, error) {
if program == nil {
return entries, nil
}
return filterEntries(entries, program)
}
func filterEntries(entries []*InternalLogEntry, program *vm.Program) ([]*InternalLogEntry, error) {
filtered := make([]*InternalLogEntry, 0, len(entries))
for _, entry := range entries {
matched, err := entryMatchesFilter(entry, program)
if err != nil {
return nil, err
}
if matched {
filtered = append(filtered, entry)
}
}
return filtered, nil
}
func entryMatchesFilter(entry *InternalLogEntry, program *vm.Program) (bool, error) {
matched, err := logfilter.Matches(program, filterRecordFromEntry(entry))
if err != nil {
return false, fmt.Errorf("filter evaluation failed: %w", err)
}
return matched, nil
}

View File

@ -0,0 +1,36 @@
package executor
import (
acl "github.com/OliveTin/OliveTin/internal/acl"
authpublic "github.com/OliveTin/OliveTin/internal/auth/authpublic"
config "github.com/OliveTin/OliveTin/internal/config"
)
func isActiveQueueEntry(entry *InternalLogEntry) bool {
return entry != nil && !entry.ExecutionFinished
}
func isQueueEntryVisible(cfg *config.Config, user *authpublic.AuthenticatedUser, entry *InternalLogEntry) bool {
if !isActiveQueueEntry(entry) || !isValidLogEntryForACL(entry) {
return false
}
return acl.IsAllowedLogs(cfg, user, entry.Binding.Action)
}
// GetActiveExecutionsACL returns unfinished executions the user may view in the queue.
func (e *Executor) GetActiveExecutionsACL(cfg *config.Config, user *authpublic.AuthenticatedUser) []*InternalLogEntry {
e.logmutex.RLock()
defer e.logmutex.RUnlock()
active := make([]*InternalLogEntry, 0)
for _, trackingID := range e.logsTrackingIdsByDate {
entry := e.logs[trackingID]
if isQueueEntryVisible(cfg, user, entry) {
active = append(active, entry)
}
}
return active
}

View File

@ -0,0 +1,72 @@
package executor
import (
"testing"
"time"
auth "github.com/OliveTin/OliveTin/internal/auth"
config "github.com/OliveTin/OliveTin/internal/config"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestGetActiveExecutionsACLFiltersFinishedAndACL(t *testing.T) {
e, cfg := testingExecutor()
allowedAction := &config.Action{
Title: "allowed",
Shell: "sleep 1",
Acls: []string{"view-logs"},
}
secretAction := &config.Action{
Title: "secret",
Shell: "sleep 1",
}
cfg.Actions = append(cfg.Actions, allowedAction, secretAction)
cfg.DefaultPermissions.Logs = false
cfg.AccessControlLists = []*config.AccessControlList{
{
Name: "view-logs",
MatchUsernames: []string{"guest"},
Permissions: config.PermissionsList{
Logs: true,
},
},
}
cfg.Sanitize()
e.RebuildActionMap()
allowedBinding := e.FindBindingWithNoEntity(allowedAction)
secretBinding := e.FindBindingWithNoEntity(secretAction)
require.NotNil(t, allowedBinding)
require.NotNil(t, secretBinding)
activeAllowed := newQueueTestLogEntry(allowedBinding, false)
finishedAllowed := newQueueTestLogEntry(allowedBinding, true)
activeSecret := newQueueTestLogEntry(secretBinding, false)
e.SetLog(activeAllowed.ExecutionTrackingID, activeAllowed)
e.SetLog(finishedAllowed.ExecutionTrackingID, finishedAllowed)
e.SetLog(activeSecret.ExecutionTrackingID, activeSecret)
user := auth.UserGuest(cfg)
active := e.GetActiveExecutionsACL(cfg, user)
require.Len(t, active, 1)
assert.Equal(t, activeAllowed.ExecutionTrackingID, active[0].ExecutionTrackingID)
}
func newQueueTestLogEntry(binding *ActionBinding, finished bool) *InternalLogEntry {
entry := &InternalLogEntry{
Binding: binding,
DatetimeStarted: time.Now(),
ExecutionTrackingID: uuid.NewString(),
ActionTitle: binding.Action.Title,
ExecutionFinished: finished,
}
if finished {
entry.DatetimeFinished = time.Now()
}
return entry
}

View File

@ -0,0 +1,178 @@
package logfilter
import (
"fmt"
"regexp"
"strings"
"github.com/expr-lang/expr"
"github.com/expr-lang/expr/vm"
)
const maxFilterLength = 512
var (
comparePattern = regexp.MustCompile(`(?i)\b(Status|Action|User|ExitCode|Blocked|TimedOut|Running)\s*(==|!=)\s*("[^"]*"|\S+)`)
containsPattern = regexp.MustCompile(`(?i)\b(Status|Action|User|Output)\s+contains\s+("[^"]*"|\S+)`)
)
// Compile parses and compiles a filter expression. Returns an error for invalid syntax.
func Compile(expression string) (*vm.Program, error) {
trimmed := strings.TrimSpace(expression)
if trimmed == "" {
return nil, nil
}
if len(trimmed) > maxFilterLength {
return nil, fmt.Errorf("filter expression exceeds maximum length of %d characters", maxFilterLength)
}
normalized, err := normalizeExpression(trimmed)
if err != nil {
return nil, err
}
return compileNormalized(normalized)
}
func compileNormalized(normalized string) (*vm.Program, error) {
return expr.Compile(normalized,
expr.Env(Record{}),
expr.AsBool(),
expr.Function("includes", includes),
expr.Function("hasTag", hasTag),
)
}
func includes(params ...any) (any, error) {
haystack, ok := params[0].(string)
if !ok {
return false, nil
}
needle, ok := params[1].(string)
if !ok {
return false, nil
}
return strings.Contains(strings.ToLower(haystack), strings.ToLower(needle)), nil
}
func hasTag(params ...any) (any, error) {
tags, ok := params[0].([]string)
if !ok {
return false, nil
}
needle, ok := params[1].(string)
if !ok {
return false, nil
}
return tagListIncludes(tags, needle), nil
}
func tagListIncludes(tags []string, needle string) bool {
needle = strings.ToLower(needle)
for _, tag := range tags {
if strings.Contains(strings.ToLower(tag), needle) {
return true
}
}
return false
}
// Matches evaluates a compiled filter against a log record.
func Matches(program *vm.Program, record Record) (bool, error) {
if program == nil {
return true, nil
}
result, err := expr.Run(program, record)
if err != nil {
return false, err
}
matched, ok := result.(bool)
if !ok {
return false, fmt.Errorf("filter expression must return a boolean")
}
return matched, nil
}
func normalizeExpression(expression string) (string, error) {
if isNegatedSearchTerm(expression) {
term := quoteLiteral(strings.TrimPrefix(expression, "!"))
return negatedSearchExpression(term), nil
}
if isPositiveSearchTerm(expression) {
return positiveSearchExpression(quoteLiteral(expression)), nil
}
normalized := replaceContainsOperators(expression)
normalized = replaceComparisons(normalized)
return replaceBooleanWords(normalized), nil
}
func isNegatedSearchTerm(expression string) bool {
if !strings.HasPrefix(expression, "!") {
return false
}
remainder := strings.TrimSpace(expression[1:])
return remainder != "" && !containsExpressionOperators(remainder)
}
func isPositiveSearchTerm(expression string) bool {
return expression != "" && !containsExpressionOperators(expression)
}
func containsExpressionOperators(expression string) bool {
lower := strings.ToLower(expression)
operators := []string{"==", "!=", "&&", "||", " contains ", "(", ")"}
for _, operator := range operators {
if strings.Contains(lower, operator) {
return true
}
}
return false
}
func negatedSearchExpression(term string) string {
return "!(" + positiveSearchExpression(term) + ")"
}
func positiveSearchExpression(term string) string {
return "includes(Action, " + term + ") || includes(User, " + term + ") || includes(Status, " + term + ") || includes(Output, " + term + ") || hasTag(Tags, " + term + ")"
}
func replaceContainsOperators(expression string) string {
return containsPattern.ReplaceAllStringFunc(expression, func(match string) string {
parts := containsPattern.FindStringSubmatch(match)
field := parts[1]
value := quoteIfNeeded(parts[2])
return fmt.Sprintf("includes(%s, %s)", field, value)
})
}
func replaceComparisons(expression string) string {
return comparePattern.ReplaceAllStringFunc(expression, func(match string) string {
parts := comparePattern.FindStringSubmatch(match)
field := parts[1]
operator := parts[2]
value := quoteIfNeeded(parts[3])
return fmt.Sprintf("%s %s %s", field, operator, value)
})
}
func replaceBooleanWords(expression string) string {
replacer := strings.NewReplacer(" and ", " && ", " AND ", " && ", " or ", " || ", " OR ", " || ")
return replacer.Replace(expression)
}
func quoteIfNeeded(value string) string {
if strings.HasPrefix(value, "\"") {
return value
}
return quoteLiteral(value)
}
func quoteLiteral(value string) string {
return "\"" + strings.ReplaceAll(value, "\"", "\\\"") + "\""
}

View File

@ -0,0 +1,50 @@
package logfilter
import (
"testing"
"github.com/expr-lang/expr/vm"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCompileNegatedSearchTerm(t *testing.T) {
program, err := Compile("!Update")
require.NoError(t, err)
assert.False(t, mustMatch(t, program, Record{Action: "Run Update script"}))
assert.True(t, mustMatch(t, program, Record{Action: "Ping host"}))
}
func TestCompileStatusNotEqual(t *testing.T) {
program, err := Compile("Status != Completed")
require.NoError(t, err)
assert.True(t, mustMatch(t, program, Record{Status: "Blocked"}))
assert.False(t, mustMatch(t, program, Record{Status: "Completed"}))
}
func TestCompileContainsAndBooleanWords(t *testing.T) {
program, err := Compile(`Status == Completed and Action contains backup`)
require.NoError(t, err)
assert.True(t, mustMatch(t, program, Record{Status: "Completed", Action: "Nightly backup"}))
assert.False(t, mustMatch(t, program, Record{Status: "Blocked", Action: "Nightly backup"}))
}
func TestCompileRejectsOverlongExpression(t *testing.T) {
_, err := Compile(string(make([]byte, maxFilterLength+1)))
require.Error(t, err)
}
func TestCompileRejectsUnknownField(t *testing.T) {
_, err := Compile(`SecretField == "x"`)
require.Error(t, err)
}
func mustMatch(t *testing.T, program *vm.Program, record Record) bool {
t.Helper()
matched, err := Matches(program, record)
require.NoError(t, err)
return matched
}

View File

@ -0,0 +1,35 @@
package logfilter
// Record exposes only log fields that may be used in filter expressions.
type Record struct {
Status string
Action string
User string
Tags []string
Blocked bool
TimedOut bool
Running bool
ExitCode int32
Output string
}
// StatusLabel matches the status text shown in the web UI.
func StatusLabel(executionFinished, blocked, timedOut, queued bool) string {
if !executionFinished {
if queued {
return "Queued"
}
return "Running"
}
return finishedStatusLabel(blocked, timedOut)
}
func finishedStatusLabel(blocked, timedOut bool) string {
if blocked {
return "Blocked"
}
if timedOut {
return "Timed out"
}
return "Completed"
}

View File

@ -0,0 +1,19 @@
# Action group concurrency
Actions may belong to one or more named groups. Each group may define a maximum number of concurrent executions shared across all actions in that group.
When a user or trigger starts an action that belongs to a group, OliveTin counts how many executions for that group are currently active. Active means the execution has been requested but not yet finished, and is not waiting in a queue.
If every configured group for that action has spare capacity, the execution proceeds through the normal execution pipeline.
If any configured group is at capacity, the new execution is queued instead of rejected. The request receives a tracking identifier immediately. The log entry shows a queued status until the execution actually starts.
Queued executions run in first-in-first-out order per OliveTin instance. When an active execution in a group finishes, OliveTin attempts to start the oldest queued execution that belongs to that group, provided all groups for that queued action now have spare capacity.
An action may belong to multiple groups. In that case, all group limits must be satisfied before the action starts or leaves the queue.
Per-action concurrency limits apply only to executions of the same action binding. When a per-action limit is exceeded, the request is blocked immediately and is not queued.
Action group concurrency limits do not survive a process restart. Queued executions that have not started are discarded when OliveTin stops.
If an action references a group name that is not defined in configuration, OliveTin logs a warning and does not apply a group limit for that name.

View File

@ -6,8 +6,8 @@
As a per-user LaunchAgent (recommended, no root needed): As a per-user LaunchAgent (recommended, no root needed):
cp app.olivetin.olivetin.plist ~/Library/LaunchAgents/ cp app.olivetin.olivetin.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/app.olivetin.olivetin.plist # start now + at login launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/app.olivetin.olivetin.plist # start now + at login
launchctl unload ~/Library/LaunchAgents/app.olivetin.olivetin.plist # stop + disable launchctl bootout gui/$(id -u) ~/Library/LaunchAgents/app.olivetin.olivetin.plist # stop + disable
As a system-wide LaunchDaemon (starts at boot, before login; needs root): As a system-wide LaunchDaemon (starts at boot, before login; needs root):
sudo cp app.olivetin.olivetin.plist /Library/LaunchDaemons/ sudo cp app.olivetin.olivetin.plist /Library/LaunchDaemons/