LZMA (Segmented)

Definitions

Kaitai Struct

Kaitai Struct is a declarative language for describing binary file formats. It allows you to define the structure of a file in a YAML-like syntax, which can then be compiled into code for various programming languages.

The following is the Kaitai Struct definition for Sony’s Segmented LZMA file format. This definition can be used to parse segmented LZMA files.

lzma.ksy


meta:
  id: lzma
  file-extension: segs, lzma
  endian: be
 
seq:
  - id: header
    type: header
  - id: segments
    type: segment
    repeat: expr
    repeat-expr: header.segment_count
 
types:
  header:
    seq:
      - id: magic
        contents: 'segs'
      - id: decompression_type
        type: u1
      - id: version
        type: u1
      - id: segment_count
        type: u2
      - id: size_inflated
        type: u4
      - id: size_deflated
        type: u4
  segment:
    seq:
      - id: size_deflated
        type: u2
      - id: size_inflated
        type: u2
      - id: offset
        type: s4
    instances:
      raw_data:
        pos: "offset - 1"
        size: "size_deflated"

Getting Started

To parse a segmented LZMA file, you can use the Kaitai Struct definition above to generate a parser in your favorite programming language.

For this guide we’ll assume you’re using Go , a fast, statically typed, opinionated language.

Install the Kaitai Struct compiler


curl -LO https://github.com/kaitai-io/kaitai_struct_compiler/releases/download/0.10/kaitai-struct-compiler_0.10_all.deb
sudo apt-get install ./kaitai-struct-compiler_0.10_all.deb

Tip

If you’re on Windows or macOS, or want the source code / binaries, you can download Kaitai Struct here.

Make a new Go project


mkdir example && cd example
go mod init example.com/m/v2
touch main.go

Code-gen the Archive struct

Download the lzma.ksy struct in our new example folder, then run:


kaitai-struct-compiler -t go lzma.ksy --go-package lzma

Write a parser

main.go


package main
 
import (
	"os"
 
	"example.com/m/v2/lzma"
 
	"github.com/kaitai-io/kaitai_struct_go_runtime/kaitai"
	"github.com/rs/zerolog"
	"github.com/rs/zerolog/log"
)
 
// Change this to the path of your segmented file
const FILE_PATH = "samples/ObjectCatalogue_5_SCEA.db"
 
func main() {
	log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})
 
	file, err := os.Open(FILE_PATH)
	if err != nil {
		log.Error().Err(err).Str("path", FILE_PATH).Msg("failed to open file")
		return
	}
 
	a := lzma.NewLzma()
	err = a.Read(kaitai.NewStream(file), nil, a)
 
	if err != nil {
		log.Error().Err(err).Msg("Failed to read segmented file.")
	}
 
	log.Info().Uint32("segmentCount", uint32(a.Header.SegmentCount)).Msg("segments info")
 
	for i := range a.Segments {
		sInf := a.Segments[i].SizeInflated
		sDef := a.Segments[i].SizeDeflated
 
		if sInf == 0 {
			sInf = 0xFFFF
		}
 
		log.Info().
			Int("index", i).
			Uint16("sizeInflated", sInf).
			Uint16("sizeDeflated", sDef).
			Float32("size%", float32(sDef)/float32(sInf)*100).
			Msg("segment")
	}
}

Run the code


go get          # Install dependencies
go run main.go  # Run