SoulsEnv

The soulsenv module provides the abstract base class for all soulsgym environments.

This SoulsEnv class includes the general gym logic and defines abstract methods that all environments have to implement.

In addition, we also provide a SoulsEnvDemo base class for demo environments. In contrast to the training environments, demos cover all phases of a boss fight and allow to demonstrate the agent’s abilities in a setting that is as close to the real game as possible.

class soulsgym.envs.soulsenv.SoulsEnv(game_speed: float = 1.0, skip_steps: bool = False)

Bases: Env, ABC

Abstract base class for soulsgym environments.

Each SoulsEnv initializes a GameInput, a GameWindow, a Game interface and a Logger to read from and to the game. During an episode the game is paused per default. At each SoulsEnv.step() call the environment applies the current input if valid (see SoulsEnv._apply_action() for details). It then unpauses the game, waits for step_size seconds and pauses again. During that time, the game runs with a speed multiplier defined by game_speed. The new game state is logged and processed to update the internal game state. The environment tries to detect and recover from any unexpected errors.

To solve any camera control issues we lock on to the boss at all times. For exceptions see SoulsEnv._lock_on().

Note

We deem it too complicated to learn from the image alone. The GameWindow class is however designed to provide this capability and the gym can easily be extended to yield image data as well.

Warning

The target game has to be running at the initialization of SoulsEnv.

Warning

Setting game_speed too high might result in unstable behaviour. The maximal value is determined by hardware factors. We strongly recommend to stick to values in the range of [1, 3].

__init__(game_speed: float = 1.0, skip_steps: bool = False)

Initialize the game managers, load the environment config and set the game properties.

Parameters:

• game_speed – Determines how fast the game runs during SoulsEnv.step().

• skip_steps – Flag to skip steps while the player is disabled.

abstract property game_id

Every Souls game has to define the base game (e.g. DarkSoulsIII, EldenRing, …).

abstract reset(seed: int | None = None, options: Any | None = None) → tuple[dict, dict]

Reset the environment to the beginning of an episode.

Parameters:

• seed – Random seed. Required by gymnasium, but does not apply to SoulsGyms.

• options – Options argument required by gymnasium. Not used in SoulsGym.

Returns:

A tuple of the first game state and the info dict after the reset.

abstract static compute_reward(game_state: GameState, next_game_state: GameState) → float

Compute the reward from the current game state and the next game state.

Parameters:

• game_state – The game state before the step.

• next_game_state – The game state after the step.

Returns:

The reward for the provided game states.

abstract property obs: Any

Return the current observation of the environment.

abstract property info: dict

Return the current info dict of the environment.

abstract game_state() → GameState

Return the current game state of the environment.

step(action: int) → tuple[dict, float, bool, bool, dict[str, Any]]

Perform a step forward in the environment with a given action.

Each step advances the ingame time by step_size seconds. The game is paused before and after the step.

Parameters:

action – The action that is applied during this step.

Returns:

A tuple of the next game state, the reward, the terminated flag, the truncated flag, and an additional info dictionary.

Raises:

ResetNeeded – step() was called after the episode was already finished.

close()

Unpause the game, reset altered game properties and reload.

Note

Does not wait for reload to complete.

current_valid_actions() → list[int]

Get the set of currently valid actions.

Returns:

An array of integers containing the currently allowed actions.

seed() → list[int]

Set the random seed for the environment.

Since we cannot control the randomness of the game and can’t precisely control the game, loop, this function does not have any effect.

Note

Setting the seed will not lead to reproducible results!

Parameters:

seed – Random seed.

Returns:

A list with 0 to comply with OpenAI’s function signature.

render()

Render the environment.

This is a no-op since we can’t render the environment and the game has to be open anyways.

_apply_action(action: int)

Apply an action to the environment.

If the player is currently in an animation where he is disabled we omit all actions. The game queues actions and performs them as soon as the player is able to move. If an agent takes action 1 during the first step while being disabled, action 1 might be executed during the next step even though the agent has chosen action 2. In particular, roll and hit commands overwrite any normal movement. As long as we can’t clear this queue we have to ensure actions are only performed when possible. Since we do not have access to the disabled game flags for the player we measure the duration of the current animation and compare that to the experimentally determined timeframe for this animation during which the player is disabled.

Parameters:

action – The action that is applied during this step.

_lock_on(target_pose: ndarray | None = None)

Reestablish lock on by orienting the camera towards the boss and pressing lock on.

If the optional target pose is given, the camera is instead oriented towards the coordinates of the target pose. If the lock on attempt fails the camera is still oriented towards the target pose. This way we imitate a lock on even if we can’t actually establish lock on.

Note

Lock on is disabled during the ThrowAtk and ThrowDef player animations because it is disabled on the game side for these animations.

Warning

Lock on actions are only queued for execution on the subsequent environment step. Since the game is paused during SoulsEnv._lock_on(), we cannot restore the lock immediately as the camera does not move.

Parameters:

target_pose – The target pose towards which the camera should be oriented from its current position.

property np_random: Generator

Returns the environment’s internal _np_random that if not set will initialise with a random seed.

Returns:

Instances of np.random.Generator

property unwrapped: Env[ObsType, ActType]

Returns the base non-wrapped environment.

Returns:

The base non-wrapped gymnasium.Env instance

Return type:

Env

class soulsgym.envs.soulsenv.SoulsEnvDemo(game_speed: float = 1.0)

Bases: SoulsEnv

Demo class to show the performance of agents in the unaltered game.

Demo envs do not reset the player and boss HP so that an episode resembles an actual boss fight. After a single episode, a hard reset of the game is necessary since either the player has died or the boss has been defeated.

__init__(game_speed: float = 1.0)

Initialize the demo environment.

Parameters:

game_speed – Determines how fast the game runs during SoulsEnv.step().

_apply_action(action: int)

Apply an action to the environment.

If the player is currently in an animation where he is disabled we omit all actions. The game queues actions and performs them as soon as the player is able to move. If an agent takes action 1 during the first step while being disabled, action 1 might be executed during the next step even though the agent has chosen action 2. In particular, roll and hit commands overwrite any normal movement. As long as we can’t clear this queue we have to ensure actions are only performed when possible. Since we do not have access to the disabled game flags for the player we measure the duration of the current animation and compare that to the experimentally determined timeframe for this animation during which the player is disabled.

Parameters:

action – The action that is applied during this step.

_lock_on(target_pose: ndarray | None = None)

Reestablish lock on by orienting the camera towards the boss and pressing lock on.

If the optional target pose is given, the camera is instead oriented towards the coordinates of the target pose. If the lock on attempt fails the camera is still oriented towards the target pose. This way we imitate a lock on even if we can’t actually establish lock on.

Note

Lock on is disabled during the ThrowAtk and ThrowDef player animations because it is disabled on the game side for these animations.

Warning

Lock on actions are only queued for execution on the subsequent environment step. Since the game is paused during SoulsEnv._lock_on(), we cannot restore the lock immediately as the camera does not move.

Parameters:

target_pose – The target pose towards which the camera should be oriented from its current position.

close()

Unpause the game, reset altered game properties and reload.

Note

Does not wait for reload to complete.

abstract static compute_reward(game_state: GameState, next_game_state: GameState) → float

Compute the reward from the current game state and the next game state.

Parameters:

• game_state – The game state before the step.

• next_game_state – The game state after the step.

Returns:

The reward for the provided game states.

current_valid_actions() → list[int]

Get the set of currently valid actions.

Returns:

An array of integers containing the currently allowed actions.

abstract property game_id

Every Souls game has to define the base game (e.g. DarkSoulsIII, EldenRing, …).

abstract game_state() → GameState

Return the current game state of the environment.

abstract property info: dict

Return the current info dict of the environment.

property np_random: Generator

Returns the environment’s internal _np_random that if not set will initialise with a random seed.

Returns:

Instances of np.random.Generator

abstract property obs: Any

Return the current observation of the environment.

render()

Render the environment.

This is a no-op since we can’t render the environment and the game has to be open anyways.

abstract reset(seed: int | None = None, options: Any | None = None) → tuple[dict, dict]

Reset the environment to the beginning of an episode.

Parameters:

• seed – Random seed. Required by gymnasium, but does not apply to SoulsGyms.

• options – Options argument required by gymnasium. Not used in SoulsGym.

Returns:

A tuple of the first game state and the info dict after the reset.

seed() → list[int]

Set the random seed for the environment.

Since we cannot control the randomness of the game and can’t precisely control the game, loop, this function does not have any effect.

Note

Setting the seed will not lead to reproducible results!

Parameters:

seed – Random seed.

Returns:

A list with 0 to comply with OpenAI’s function signature.

step(action: int) → tuple[dict, float, bool, bool, dict[str, Any]]

Perform a step forward in the environment with a given action.

Each step advances the ingame time by step_size seconds. The game is paused before and after the step.

Parameters:

action – The action that is applied during this step.

Returns:

A tuple of the next game state, the reward, the terminated flag, the truncated flag, and an additional info dictionary.

Raises:

ResetNeeded – step() was called after the episode was already finished.

property unwrapped: Env[ObsType, ActType]

Returns the base non-wrapped environment.

Returns:

The base non-wrapped gymnasium.Env instance

Return type:

Env