FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat: add a maximum hold duration to expiring locks by jeswr · Pull Request #2221 · CommunitySolidServer/CommunitySolidServer · GitHub

Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension .json  (1) .md  (2) .ts  (2) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
4 changes: 4 additions & 0 deletions RELEASE_NOTES.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
- `AsyncHandler` and utility handlers such as `WaterfallHandler`: [asynchronous-handlers](https://www.npmjs.com/package/asynchronous-handlers).
- Utilities for creating vocabularies: [rdf-vocabulary](https://www.npmjs.com/package/rdf-vocabulary).
- WAC/ACP authorization: [@solidlab/policy-engine](https://github.com/CommunitySolidServer/policy-engine).
- Expiring read/write lockers can enforce an optional maximum hold duration,
independent of activity-based lock renewals.

### Data migration

Expand All @@ -28,6 +30,8 @@ The following changes pertain to the imports in the default configs:
- There is a new import option for `storage/middleware`: `cache.json`, which adds caching for backend resources.
All default configurations have been changed to use that option.
Servers using worker threads can not use this option.
- There is a new opt-in `util/resource-locker/file-capped.json` configuration that caps file-based locks at one hour.
Existing resource locker configurations remain uncapped.

The following changes are relevant for v7 custom configs that replaced certain features.

Expand Down
1 change: 1 addition & 0 deletions config/util/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Which locking mechanism to use to for example prevent 2 write simultaneous write

* *debug-void*: No locking mechanism, does not prevent simultaneous read/writes.
* *file*: Uses a file-system based locking mechanism (process-safe/thread-safe).
* *file-capped*: Uses the file-system locker with a one-hour maximum hold duration.
* *memory*: Uses an in-memory locking mechanism.
* *redis*: Uses a Redis store for locking that supports threadsafe read-write locking (process-safe/thread-safe).

Expand Down
48 changes: 48 additions & 0 deletions config/util/resource-locker/file-capped.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
{
"@context": [
"https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^8.0.0/components/context.jsonld",
"https://linkedsoftwaredependencies.org/bundles/npm/asynchronous-handlers/^1.0.0/components/context.jsonld"
],
"@graph": [
{
"comment": [
"Allows multiple simultaneous read operations. Locks are stored on filesystem. This locker is threadsafe.",
"Locks expire after inactivity, or 1 hour after acquisition, so continually renewed locks can not block a resource forever."
],
Comment thread
jeswr marked this conversation as resolved.
"@id": "urn:solid-server:default:ResourceLocker",
"@type": "WrappedExpiringReadWriteLocker",
"locker": {
"@type": "PartialReadWriteLocker",
"locker": {
"@id": "urn:solid-server:default:FileSystemResourceLocker",
"@type": "FileSystemResourceLocker",
"args_rootFilePath": { "@id": "urn:solid-server:default:variable:rootFilePath" }
}
},
"expiration": 6000,
"maxHoldDuration": 3600000
},
{
"@id": "urn:solid-server:default:CleanupInitializer",
"@type": "SequenceHandler",
"handlers": [
{
"comment": "Makes sure the FileSystemResourceLocker starts with a clean slate when the application is started.",
"@type": "InitializableHandler",
"initializable": { "@id": "urn:solid-server:default:FileSystemResourceLocker" }
}
]
},
{
"@id": "urn:solid-server:default:CleanupFinalizer",
"@type": "SequenceHandler",
"handlers": [
{
"comment": "Makes sure the lock folder is removed when the application stops.",
"@type": "FinalizableHandler",
"finalizable": { "@id": "urn:solid-server:default:FileSystemResourceLocker" }
}
]
}
]
}
43 changes: 39 additions & 4 deletions src/util/locking/WrappedExpiringReadWriteLocker.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,19 @@ export class WrappedExpiringReadWriteLocker implements ExpiringReadWriteLocker {

protected readonly locker: ReadWriteLocker;
protected readonly expiration: number;
protected readonly maxHoldDuration: number;

/**
* @param locker - Instance of ResourceLocker to use for acquiring a lock.
* @param expiration - Time in ms after which the lock expires.
* @param expiration - Time in ms after which the lock expires due to inactivity.
* @param maxHoldDuration - Maximum total time in ms a lock can be held, independent of activity renewals,
* to prevent continually renewed locks from blocking a resource forever.
* `0`, the default, disables the cap.
*/
public constructor(locker: ReadWriteLocker, expiration: number) {
public constructor(locker: ReadWriteLocker, expiration: number, maxHoldDuration = 0) {
this.locker = locker;
this.expiration = expiration;
this.maxHoldDuration = maxHoldDuration;
}

public async withReadLock<T>(
Expand All @@ -48,32 +53,62 @@ export class WrappedExpiringReadWriteLocker implements ExpiringReadWriteLocker {
whileLocked: (maintainLock: () => void) => PromiseOrValue<T>,
): Promise<T> {
let timer: Timeout;
let maxTimer: Timeout | undefined;
let createTimeout: () => Timeout;
let active = true;

function clearTimers(): void {
clearTimeout(timer);
if (maxTimer) {
clearTimeout(maxTimer);
maxTimer = undefined;
}
}

// Promise that throws an error when the timer finishes
const timerPromise = new Promise<never>((resolve, reject): void => {
// Starts the timer that will cause this promise to error after a given time
createTimeout = (): Timeout => setTimeout((): void => {
active = false;
clearTimers();
this.logger.error(`Lock expired after ${this.expiration}ms on ${identifier.path}`);
reject(new InternalServerError(`Lock expired after ${this.expiration}ms on ${identifier.path}`));
}, this.expiration);

timer = createTimeout();

// Absolute deadline on the total hold time, which renewals do not extend
if (this.maxHoldDuration > 0) {
maxTimer = setTimeout((): void => {
active = false;
clearTimers();
this.logger.warn(
`Lock reached its maximum hold duration of ${this.maxHoldDuration}ms on ${identifier.path}`,
);
reject(new InternalServerError(
`Lock reached its maximum hold duration of ${this.maxHoldDuration}ms on ${identifier.path}`,
));
}, this.maxHoldDuration);
}
});

// Restarts the timer
const renewTimer = (): void => {
if (!active) {
return;
}
this.logger.verbose(`Renewed expiring lock on ${identifier.path}`);
clearTimeout(timer);
timer = createTimeout();
};

// Runs the main function and cleans up the timer afterwards
// Runs the main function and cleans up the timers afterwards
async function runWithTimeout(): Promise<T> {
try {
return await whileLocked(renewTimer);
} finally {
clearTimeout(timer);
active = false;
clearTimers();
}
}

Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import type { ResourceIdentifier } from '../../../../src/http/representation/ResourceIdentifier';
import { EqualReadWriteLocker } from '../../../../src/util/locking/EqualReadWriteLocker';
import { MemoryResourceLocker } from '../../../../src/util/locking/MemoryResourceLocker';
import type { ReadWriteLocker } from '../../../../src/util/locking/ReadWriteLocker';
import { WrappedExpiringReadWriteLocker } from '../../../../src/util/locking/WrappedExpiringReadWriteLocker';
import type { PromiseOrValue } from '../../../../src/util/PromiseUtil';
Expand Down Expand Up @@ -29,6 +31,10 @@ describe('A WrappedExpiringReadWriteLocker', (): void => {
locker = new WrappedExpiringReadWriteLocker(wrappedLocker, expiration);
});

afterEach((): void => {
jest.clearAllTimers();
});

it('calls the wrapped locker for locking.', async(): Promise<void> => {
let prom = locker.withReadLock(identifier, syncCb);
await expect(prom).resolves.toBe('sync');
Expand Down Expand Up @@ -88,4 +94,115 @@ describe('A WrappedExpiringReadWriteLocker', (): void => {
jest.advanceTimersByTime(5000);
await expect(prom).rejects.toThrow(`Lock expired after ${expiration}ms on ${identifier.path}`);
});

it('does not cap the total hold time when maxHoldDuration defaults to 0.', async(): Promise<void> => {
async function refreshCb(maintainLock: () => void): Promise<string> {
return new Promise((resolve): any => {
setTimeout(maintainLock, 750);
setTimeout(maintainLock, 1500);
setTimeout(maintainLock, 2250);
setTimeout((): void => resolve('refresh'), 2900);
});
}
const prom = locker.withReadLock(identifier, refreshCb);
jest.advanceTimersByTime(2900);
await expect(prom).resolves.toBe('refresh');
});

it('does not cap the total hold time when maxHoldDuration is explicitly 0.', async(): Promise<void> => {
locker = new WrappedExpiringReadWriteLocker(wrappedLocker, expiration, 0);
async function refreshCb(maintainLock: () => void): Promise<string> {
return new Promise((resolve): any => {
setTimeout(maintainLock, 750);
setTimeout(maintainLock, 1500);
setTimeout(maintainLock, 2250);
setTimeout((): void => resolve('refresh'), 2900);
});
}
const prom = locker.withReadLock(identifier, refreshCb);
jest.advanceTimersByTime(2900);
await expect(prom).resolves.toBe('refresh');
});

it('rejects once the maximum hold duration is exceeded, even while being renewed.', async():
Promise<void> => {
const maxHoldDuration = 2000;
locker = new WrappedExpiringReadWriteLocker(wrappedLocker, expiration, maxHoldDuration);
async function trickleCb(maintainLock: () => void): Promise<void> {
return new Promise((resolve): any => {
setTimeout(maintainLock, 750);
setTimeout(maintainLock, 1500);
setTimeout(resolve, 10000);
});
}
const prom = locker.withWriteLock(identifier, trickleCb);
jest.advanceTimersByTime(maxHoldDuration);
await expect(prom).rejects
.toThrow(`Lock reached its maximum hold duration of ${maxHoldDuration}ms on ${identifier.path}`);
Comment thread
jeswr marked this conversation as resolved.
});

it('releases the underlying resource lock once the maximum hold duration is exceeded.', async(): Promise<void> => {
const maxHoldDuration = 2000;
const resourceLocker = new MemoryResourceLocker();
const release = jest.spyOn(resourceLocker, 'release');
locker = new WrappedExpiringReadWriteLocker(
new EqualReadWriteLocker(resourceLocker),
expiration,
maxHoldDuration,
);
let indicateLockAcquired: () => void;
const lockAcquired = new Promise<void>((resolve): void => {
indicateLockAcquired = resolve;
});

const prom = locker.withWriteLock(identifier, async(maintainLock): Promise<never> => {
indicateLockAcquired();
setTimeout(maintainLock, 750);
setTimeout(maintainLock, 1500);
return new Promise<never>((): void => undefined);
});
await lockAcquired;
jest.advanceTimersByTime(maxHoldDuration);

await expect(prom).rejects
.toThrow(`Lock reached its maximum hold duration of ${maxHoldDuration}ms on ${identifier.path}`);
expect(release).toHaveBeenCalledTimes(1);
expect(release).toHaveBeenCalledWith(identifier);
});

it('ignores renewals after the maximum hold duration is exceeded.', async(): Promise<void> => {
const maxHoldDuration = 2000;
locker = new WrappedExpiringReadWriteLocker(wrappedLocker, expiration, maxHoldDuration);
let maintainLock: (() => void) | undefined;
const prom = locker.withWriteLock(identifier, async(renew): Promise<never> => {
maintainLock = renew;
setTimeout(renew, 750);
setTimeout(renew, 1500);
return new Promise<never>((): void => undefined);
});
jest.advanceTimersByTime(maxHoldDuration);
await expect(prom).rejects
.toThrow(`Lock reached its maximum hold duration of ${maxHoldDuration}ms on ${identifier.path}`);
expect(jest.getTimerCount()).toBe(0);

expect(maintainLock).toBeDefined();
maintainLock?.();
expect(jest.getTimerCount()).toBe(0);
});

it('still allows renewals up to the maximum hold duration.', async(): Promise<void> => {
const maxHoldDuration = 5000;
locker = new WrappedExpiringReadWriteLocker(wrappedLocker, expiration, maxHoldDuration);
async function refreshCb(maintainLock: () => void): Promise<string> {
return new Promise((resolve): any => {
setTimeout(maintainLock, 750);
setTimeout(maintainLock, 1500);
setTimeout(maintainLock, 2250);
setTimeout((): void => resolve('refresh'), 3000);
});
}
const prom = locker.withReadLock(identifier, refreshCb);
jest.advanceTimersByTime(3000);
await expect(prom).resolves.toBe('refresh');
});
});
Loading

Back | FazBrowse Home | New Git URL