Skip to Content
Nextra 4.0 is released 🎉
FormatsArchive

Archive

PlayStation Home uses a proprietary archive format known as BAR, to store assets like Scenes, Objects.

Note

It’s unclear what “BAR” might stand for. A common guess is “Binary ARchive”, but there’s no official sources.

Locations

They are mainly found in two places:

  1. In the game’s files (<root>/dev_hdd0/game/<NPID>)
  2. In the CDN, wrapped inside SDAT containers.

Versions

BAR (version 1)

BAR is the first version of the archive, and it’s directly deserializable by a compatible parser.

The Header and Table of Contents are stored in plaintext, and the entries are (optionally) compressed and encrypted.

Cryptography

  • Algorithms: Blowfish
  • Keys: Signature and Default
  • IVs: self-contained, stored in the entries’ metadata

Each entry is composed of two parts:

  1. A head (4 + 20 bytes), containing the head’s FOURCC and the file’s SHA-1 hash.
  2. A body, containing the body’s FOURCC and the file’s actual contents.

They are encrypted separately:

  1. The head is encrypted using the Signature key.
  2. The body’s contents is encrypted using the Default key.
Important

The body’s FOURCC is left as plaintext! It does NOT get encrypted.

The final entry data is, therefore:

const data = head + body_fourcc + body;

In order to decrypt the actual file, we must skip the first 28 bytes and decrypt the body’s contents directly with the Default key.

The IV for each entry is calculated by concatenating different pieces of information from the archive and the entries.

Below is a Rust snipped to correctly calculate the IV:

pub const HEADER_SIZE: usize = 20;
pub const FILE_ENTRY_SIZE: usize = 16;
 
fn forge_iv(
    num_files: u64,           // Total number of files in the archive
    uncompressed_size: u64,   // Uncompressed size of the entry
    compressed_size: u64,     // Compressed size of the entry
    offset: u64,              // Offset of the entry in the archive
    timestamp: i32,           // Timestamp of the archive
) -> [u8; 8] {
    let entries_data_len = FILE_ENTRY_SIZE * num_files as usize; // Length of the entries data
    let file_offset = HEADER_SIZE + entries_data_len + offset as usize; // Where the specific file's data starts
 
    let extended_timestamp = 0xFFFFFFFF00000000 | (timestamp as u64);
 
    let iv = {
        let uncompressed_size = (uncompressed_size as u64) << 0x30;
        let compressed_size = ((compressed_size & 0xFFFF) as u64) << 0x20;
        let file_offset = ((file_offset & 0x3FFFC) as u64) << 0xE;
        let extended_timestamp = extended_timestamp & 0xFFFF;
 
        uncompressed_size | compressed_size | file_offset | extended_timestamp
    };
 
    // IV is always big-endian, becuase the XTEA algorithm itself is big-endian.
    iv.to_be_bytes()
}

SHARC (version 2)

  • Algorithms: XTEA and AES
  • Keys: Signature and Default
  • IVs: self-contained, stored in the entries’ metadata

SHARC is an evolution of BAR, also sometimes internally referred as “BAR version 2”.

It aims to enhance security and confidentiality by:

  1. Encrypting the Header (priority, timestamp and entry_count) with AES, using a static key and a self-contained IV.
  2. Encrypting the Table of Contents (entry list) with AES, using a Header-contained key and the same IV from before (increased by 1).

Additionally, each entry’s IV is now self-contained and encrypted with XTEA in Counter (CTR) mode.

Ironically, SHARC entries are much simpler to decode and decrypt than BAR entries.

Mapping

The archives do not retain the paths or names of the files they contain. Instead, they store a CRC32 hash of the file’s path, relative to the archive’s root. This means that in order to recover a file’s original path, you must perform a process known as mapping.

It roughly works as follows:

  1. Decrypt and extract every entry from the archive.
  2. Scan every file using RegEx patterns to find possible references to paths in the files.
  3. Hash every found path with the Home CRC algorithm.
  4. Construct a map between the computed hashes and their original paths.

Then, when a file is extracted from the archive, its hash can be looked up in the mapping table to find its original path.

This is obviously a best-effort approach, as it relies on the patterns found in the files. Some paths may not be recoverable if they were not referenced in any file.

Definitions

Kaitai Struct

Kaitai Struct  is a declarative language for describing binary file formats. It allows you to define the structure of a file in a YAML-like syntax, which can then be compiled into code for various programming languages.

The following is the Kaitai Struct definition for the PlayStation Home archive format, which supports both BAR and SHARC versions. This definition can be used to parse and extract information from these archives.

Warning

Before using this definition for SHARC archives, make sure to decrypt the archive’s Header and TOC first!

archive.ksy
meta:
  id: archive
  title: "PlayStation Home Archive"
  file-extension:
    - bar
    - sharc
 
seq:
  - id: magic
    size: 4
    doc: |
      Magic number identifying the archive. Must be either big-endian or little-endian signature.
    valid:
      any-of:
        - "[0xAD, 0xEF, 0x17, 0xE1]"
        - "[0xE1, 0x17, 0xEF, 0xAD]"
 
  - id: info
    type: info
    doc: |
      The first 4 bytes after the magic, containing both flags (high 16 bits) and version (low 16 bits).
 
      On SHARC archives (version 512), this also contains the IV to decrypt the header.
 
  - id: header
    type: header
 
  - id: entries
    type: entry
    repeat: expr
    repeat-expr: header.entry_count
    doc: |
      The entries in the archive.
 
      The number of entries is determined by the `entry_count` field in the header.
      Each entry contains information about a file in the archive, including its name,
      offset, and size.
 
      WARNING: since SHARC archives are encrypted, the entries will not be readable
      until the archive is decrypted. Therefore, this field is replaced by `entries_data`
 
types:
  info:
    doc: |
      Information about the archive, including its version and flags.
 
      In SHARC archives (version 512), this also contains the IV to decrypt
      the header.
    meta:
      endian:
        switch-on: _root.magic
        cases:
          "[0xAD, 0xEF, 0x17, 0xE1]": be
          "[0xE1, 0x17, 0xEF, 0xAD]": le
    seq:
      - id: flags_and_version
        type: u4
        doc: |
          The first 4 bytes after the magic, containing both flags (high 16 bits) and version (low 16 bits).
        valid:
          expr: (flags_and_version >> 16) == 256 or (flags_and_version >> 16) == 512
 
      - id: iv
        size: 16
        if: version == 512
        doc: |
          AES IV used for encryption.
 
          This field is only present on SHARC archives (version 512).
 
    instances:
      flags:
        value: flags_and_version & 0xFFFF
        doc: |
          Archive flags (upper 16 bits of flags_and_version).
      flag_ztoc:
        value: (flags & 0b0001) != 0
        doc: |
          `ZTOC: 0b0001`
 
          This flag indicates that the archive's Table of Contents (TOC) is compressed
          with Zlib.
      flag_leanzlib:
        value: (flags & 0b0010) != 0
        doc: |
          `LEAN_ZLIB: 0b0010`
 
          It is currently unknown what this flag does, but it is always set to `0` in
          every original archive encountered.
 
          Perhaps it would strip the header of the Zlib stream, but this is not confirmed.
      version:
        value: flags_and_version >> 16
        doc: |
          Archive version (lower 16 bits of flags_and_version).
 
  header:
    meta:
      endian:
        switch-on: _root.magic
        cases:
          "[0xAD, 0xEF, 0x17, 0xE1]": be
          "[0xE1, 0x17, 0xEF, 0xAD]": le
    seq:
      - id: priority
        type: s4
        doc: |
          Priority of the archive.
 
          If two archives have conflicting paths, the game will pick which one to prefer
          based on whichever has the lowest priority.
 
      - id: timestamp
        type: s4
        doc: |
          Contrary to what the name suggests, this field does not represent the
          timestamp of the archive itself, but rather seems to be a random 4-byte value
          used by the game to validate the archive's integrity.
 
          This value is read by the game and compared against a known good value, retrieved
          from either a Scene List or an Object Catalogue.
 
      - id: entry_count
        type: u4
        doc: |
          The number of entries in the archive.
 
      - id: key
        size: 16
        if: _root.info.version == 512
        doc: |
          AES key used for encryption.
 
          This field is only present on SHARC archives (version 512).
 
  entry:
    meta:
      endian:
        switch-on: _root.magic
        cases:
          "[0xAD, 0xEF, 0x17, 0xE1]": be
          "[0xE1, 0x17, 0xEF, 0xAD]": le
    seq:
      - id: name
        type: s4
        doc: |
          CRC hash of the entry's path in the archive, relative to the archive's root.
 
          As the name is not stored in the archive, this inherently makes the archive
          lose information.
 
          Because of this, in order to recover the lost information, the archive
          must undergo a process commonly referred to As "mapping", which involves
          scanning the entire archive's contents, building a map of every possible
          path detected, and then hashing them with the same CRC algorithm, to
          do something similar to a rainbow table attack, recovering the original
          file path on a best-effort basis.
 
      - id: offset_comp
        type: u4
        doc: |
          A 32-bit unsigned integer representing the offset and the compression type
          used by the entry.
 
          Note that the offset refers to the archive's `data` section, and not the
          overall position in the file. As such, the first entry's offset will always
          be a low value.
 
          The offset is aligned to 4-bytes, represented by the integer's 30 most significant
          bits.
 
          The compression type is represented by the integer's 2 least significant bits,
          which allow for a total of 4 different compression types.ession_type = offset_comp & 0b11
 
      - id: size_inflated
        type: u4
        doc: |
          The size of the entry after inflation.
 
      - id: size_deflated
        type: u4
        doc: |
          The size of the entry after deflation.
 
      - id: iv
        size: 8
        if: _root.info.version == 512
        doc: |
          XTEA IV used for encryption.
 
          This field is only present on SHARC archives (version 512).
 
    instances:
      offset:
        value: offset_comp & 0xFFFFFFFC
        doc: |
          Actual offset into the archive's `data` section,
          aligned to 4 bytes.
      compression_type:
        value: offset_comp & 0x3
        enum: compression_type_enum
        doc: |
          Compression type used for this entry.
          The 2 least significant bits of `offset_comp`.
      raw_data:
        pos: "(_root.info.version == 256 ? (20 + (16 * _root.header.entry_count) + offset) : (52 + (24 * _root.header.entry_count) + offset))"
        size: "((size_deflated + 3) & ~3)"
 
enums:
  compression_type_enum:
    0: none
    1: zlib
    2: edge_zlib
    3: encrypted

Getting Started

To parse an Archive, you can use the Kaitai Struct definition above to generate a parser in your favorite programming language.

For this guide we’ll assume you’re using Go , a fast, statically typed, opinionated language.

Install the Kaitai Struct compiler

curl -LO https://github.com/kaitai-io/kaitai_struct_compiler/releases/download/0.10/kaitai-struct-compiler_0.10_all.deb
sudo apt-get install ./kaitai-struct-compiler_0.10_all.deb
Tip

If you’re on Windows or macOS, or want the source code / binaries, you can download  Kaitai Struct here.

Make a new Go project

mkdir example && cd example
go mod init example.com/m/v2
touch main.go

Code-gen the Archive struct

Download the archive.ksy struct in our new example folder, then run:

kaitai-struct-compiler -t go archive.ksy --go-package archive

Write a parser

main.go
package main
 
import (
    "example.com/m/v2/archive"
	"encoding/json"
	"os"
 
	"github.com/kaitai-io/kaitai_struct_go_runtime/kaitai"
	"github.com/rs/zerolog"
	"github.com/rs/zerolog/log"
)
 
// Change this to the path of your BAR / decrypted SHARC archive
const COREDATA_PATH = "samples/COREDATA.BAR"
 
func main() {
	log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})
 
	file, err := os.Open(COREDATA_PATH)
	if err != nil {
		log.Error().Err(err).Str("path", COREDATA_PATH).Msg("failed to open file")
		return
	}
 
	a := archive.NewArchive()
	err = a.Read(kaitai.NewStream(file), nil, a)
 
	if err != nil {
		log.Error().Err(err).Msg("Failed to read archive. If it's a SHARC, please make sure it's been decrypted properly first!")
	}
 
	log.Info().Uint32("entryCount", a.Header.EntryCount).Msg("archive info")
 
	for i := range a.Entries {
		sInf := a.Entries[i].SizeInflated
		sDef := a.Entries[i].SizeDeflated
 
		log.Info().
			Int("index", i).
			Uint32("sizeInflated", sInf).
			Uint32("sizeDeflated", sDef).
			Float32("size%", float32(sDef)/float32(sInf)*100).
			Msg("entry")
	}
}

Run the code

Warning

Go support for Kaitai Struct is still experimental. The compiler may complain quite a bit!

go get          # Install dependencies
go run main.go  # Run
Last updated on
ScenesSDAT