From luke at foolishgames.com Sun Jun 29 12:16:23 2008 From: luke at foolishgames.com (Lucas Holt) Date: Sun, 29 Jun 2008 12:16:23 -0400 Subject: [Midnightbsd-mports] Naming packages: LATEST_LINK Message-ID: I've been working on a new solution to generating the symbolic links used on the FTP server for package names. When someone creates packages in mports, they are stored inside a directory like so: /usr/mports/Packages/${ARCH}/All Then symlinks are created in Latest/ as well as for each main category listed in the port makefile. The ftp server setup for pkg_add expects to have Latest defined, and by convention FreeBSD likes to also create the symlinks for each category to make it easier on users looking through the FTP for packages. The Latest/ subdirectory is required, but the others are not. Magus, on the other hand, generates packages and uploads them to a central location using scp. They are stored with their full package name on the ftp server. My first attempt was to create a script that could take the list of packages `ls` and generate symlinks inside the Latest directory. This had problems. It's not easy to parse through a string looking for the last - in a script. Names can be like xorg-server-7.3.0_1.tbz and vim5-5.0.8.tbz The former would not work properly, but the latter would. Chris created another script for this purpose that traversed the mports tree at /usr/mports and generated the symlinks at a specific path on the file system. So I could run this on stargazer and it new the exact package names of every port . This had the benefit of using the same logic that the ports system used, but made absolute paths which did not work with the FTP server. so instead of Latest/ vim5.tbz -> ../All/vim5-5.0.8.tbz, we'd get Latest/vim5.tbz -> /home/ ftp/pub/MidnightBSD/mports/Packages/i386/current/All/ vim5-5.0.8.tbz This would result on pkg_add failing, etc. My next approach was to use the data from Magus itself for a given run. Two fields are stored, pkgname and version in the ports table. I wrote a simple program to do a select on the table for those two fields on ports that did not fail, internal, or were marked untested. (another words pass and warn ports) This looked good at first. I assumed pkgname was in fact the actual package name. This is not the case. Some ports have some interesting hacks in the names. The ports system has a LATEST_LINK variable which sets the name of the package generated. For instance, the vim5 port has LATEST_LINK = vim5. This means that vim5 and vim7 end up as unique package names even though the port itself is just named vim. With my method above, the result is vim.tbz for Latest/ which is incorrect and causes a collision with vim5 and vim7. Other ports have a setting for no latest link. To make matters even more complex, some ports append or prepend to the PKGNAME using PKGNAMESUFFIX and PKGNAMEPREFIX (as well as PKGNAMESUFFIX2). In most cases, it appears the pkgname is accurate in the database aside from the LATEST_LINK situation. I see a few solutions to this problem. 1.) I just write a quick program to parse the package names properly. A for loop going through characters would work fine. One thing I'm unclear about is how the dependancies are stored with packages though. For instance, does a meta port store the LATEST_LINK name or the pkgname as the depends? 2.) We start storing LATEST_LINK when the magus indexer runs in the database. This would allow me to use that in my program. Assumes LATEST_LINK is the determinate for other packages dependancy listing. 3.) We get rid of LATEST_LINK and name packages using another convention in these cases. Maybe set PORTNAME = vim and PKGNAMESUFFIX = 5? This could be a problem for ports with multiple languages or that append threads. We might need PKGNAMESUFFIX3 and/or PKGLANG which gets appended with the package language choice (de, ru, ...) Magus would need to be aware of this. Normally, I'd just implement the first choice without discussing it. However, at some point mport will be using it's own index (sqlite) to store information and I don't know what schema we're using for some of these details. I was under the impression magus would be generating the index on a run. If that is the case, magus will need to know the correct package name and possibly a new convention or some variant of what we have now would be used for naming. Is apache20 apache-2.0.54 or apache20-2.0.54 in the new convention? I plan on putting up some 0.2 packages soon. This would be helpful to get resolved before we do the next release. Lucas Holt Luke at FoolishGames.com ________________________________________________________ MidnightBSD.org (Free OS) JustJournal.com (Free blogging)