[Midnightbsd-cvs] [MidnightBSD/mports] dd9c05: ai/pytorch: add optional distributed training support

[Lucas Holt]

  Branch: refs/heads/ai/pytorch-distributed
  Home:   https://github.com/MidnightBSD/mports
  Commit: dd9c056fdf78242e218bbd6f5f3c793cdab858f5
      https://github.com/MidnightBSD/mports/commit/dd9c056fdf78242e218bbd6f5f3c793cdab858f5
  Author: Lucas Holt <luke at foolishgames.com>
  Date:   2026-09-07 (Mon, 07 Sep 2026)

  Changed paths:
    M ai/pytorch/Makefile
    A ai/pytorch/files/patch-third__party_tensorpipe_tensorpipe_common_system.cc
    M ai/pytorch/pkg-plist

  Log Message:
  -----------
  ai/pytorch: add optional distributed training support

Add a DISTRIBUTED option, off by default, that turns on USE_DISTRIBUTED and
builds the gloo, MPI and tensorpipe backends.

Upstream disables USE_DISTRIBUTED on every platform that is not Linux or
Windows (CMakeLists.txt:200, "not tested and likely won't work without
additional changes"), and the FreeBSD port leaves it that way. Two things are
needed to make it build here:

- The bundled libuv gets no BSD sources at all, because CMAKE_SYSTEM_NAME is
  "MidnightBSD" and libuv tests for FreeBSD. The patch for that is already in
  the tree; this change is what finally exercises it. libuv is required rather
  than optional: gloo's other transport is tcp, which is Linux-only because it
  uses epoll(2) and which gloo gates on CMAKE_SYSTEM_NAME STREQUAL "Linux", so
  without libuv gloo would be built with no transport at all. USE_LIBUV is
  therefore tied to the same knob instead of being separately selectable.

- tensorpipe defines getBootIDInternal() only for __APPLE__ and __linux__, so
  common/system.cc does not compile here. The new patch adds a __FreeBSD__
  branch. FreeBSD and MidnightBSD have no per-boot UUID, so it combines the
  stable kern.hostuuid with kern.boottime, which matches how tensorpipe uses
  the boot ID: deciding whether two processes share a machine and a boot for
  shared-memory channel eligibility.

The option adds 26 installed files, all tensorpipe: 22 headers,
libtensorpipe.a, libtensorpipe_uv.a and two cmake config files. gloo and libuv
install nothing, they are linked statically into libtorch_cpu.

Verified on MidnightBSD 4.2 amd64 with the option enabled: full build clean,
and libtorch_cpu.so exports 782 distributed symbols including ProcessGroupGloo,
ProcessGroupMPI and TensorPipeAgent. Packaged as pytorch-2.13.0_1.

Only the build and link are verified. Distributed training has not been
exercised at runtime, which needs multiple processes or hosts, and the libuv
gloo transport is far less travelled than the Linux tcp one. That is why the
option is off by default.

AI-Assisted-by: Claude Opus 5 (1M context) <noreply at anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
Claude-Session: https://claude.ai/code/session_01XkHF8bn8Ehit2aSoWjCshm
Signed-off-by: Lucas Holt <luke at foolishgames.com>



To unsubscribe from these emails, change your notification settings at https://github.com/MidnightBSD/mports/settings/notifications