[Midnightbsd-cvs] mports [15451] svnadmin/hooks: add several validation scripts
laffer1 at midnightbsd.org
laffer1 at midnightbsd.org
Mon Sep 16 22:04:28 EDT 2013
Revision: 15451
http://svnweb.midnightbsd.org/mports/?rev=15451
Author: laffer1
Date: 2013-09-16 22:04:27 -0400 (Mon, 16 Sep 2013)
Log Message:
-----------
add several validation scripts
Added Paths:
-----------
svnadmin/hooks/pre-commit
svnadmin/hooks/scripts/case-insensitive.py
svnadmin/hooks/scripts/deny-filenames.sh
svnadmin/hooks/scripts/detect-merge-conflicts.sh
svnadmin/hooks/scripts/detect-nonewline-at-eof.sh
svnadmin/hooks/scripts/log-police.py
svnadmin/hooks/scripts/mergeinfo.sh
svnadmin/hooks/scripts/verify.py
Added: svnadmin/hooks/pre-commit
===================================================================
--- svnadmin/hooks/pre-commit (rev 0)
+++ svnadmin/hooks/pre-commit 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,94 @@
+#!/bin/sh
+
+# $MidnightBSD$
+
+# PRE-COMMIT HOOK
+#
+# The pre-commit hook is invoked before a Subversion txn is
+# committed. Subversion runs this hook by invoking a program
+# (script, executable, binary, etc.) named 'pre-commit' (for which
+# this file is a template), with the following ordered arguments:
+#
+# [1] REPOS-PATH (the path to this repository)
+# [2] TXN-NAME (the name of the txn about to be committed)
+#
+# The default working directory for the invocation is undefined, so
+# the program should set one explicitly if it cares.
+#
+# If the hook program exits with success, the txn is committed; but
+# if it exits with failure (non-zero), the txn is aborted, no commit
+# takes place, and STDERR is returned to the client. The hook
+# program can use the 'svnlook' utility to help it examine the txn.
+#
+# On a Unix system, the normal procedure is to have 'pre-commit'
+# invoke other programs to do the real work, though it may do the
+# work itself too.
+#
+# *** NOTE: THE HOOK PROGRAM MUST NOT MODIFY THE TXN, EXCEPT ***
+# *** FOR REVISION PROPERTIES (like svn:log or svn:author). ***
+#
+# This is why we recommend using the read-only 'svnlook' utility.
+# In the future, Subversion may enforce the rule that pre-commit
+# hooks should not modify the versioned data in txns, or else come
+# up with a mechanism to make it safe to do so (by informing the
+# committing client of the changes). However, right now neither
+# mechanism is implemented, so hook writers just have to be careful.
+#
+# Note that 'pre-commit' must be executable by the user(s) who will
+# invoke it (typically the user httpd runs as), and that user must
+# have filesystem-level permission to access the repository.
+#
+# On a Windows system, you should name the hook program
+# 'pre-commit.bat' or 'pre-commit.exe',
+# but the basic idea is the same.
+#
+# The hook program typically does not inherit the environment of
+# its parent process. For example, a common problem is for the
+# PATH environment variable to not be set to its usual value, so
+# that subprograms fail to launch unless invoked via absolute path.
+# If you're having unexpected problems with a hook program, the
+# culprit may be unusual (or missing) environment variables.
+#
+# Here is an example hook script, for a Unix /bin/sh interpreter.
+# For more examples and pre-written hooks, see those in
+# the Subversion repository at
+# http://svn.collab.net/repos/svn/trunk/tools/hook-scripts/ and
+# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/
+
+
+REPO="$1"
+TXN="$2"
+
+. /home/svn/repos/mports/hooks/scripts/env.sh
+
+# POLICY: mime-type must be unset, text/*, application/* or image/*
+# POLICY: if a file does has mnbsd:nokeywords, then svn:keywords must not be set
+# POLICY: if a file has binary chars and no mnbsd:notbinary, then pretend its not binary
+# POLICY: if a file is binary, then it must have mime application/* or image/*
+# POLICY: if a file does not have mnbsd:nokeywords, or is binary then svn:keywords must be set
+# POLICY: if svn:keywords is set, $ MidnightBSD $ must be present and condensed.
+# POLICY: If a file has text/*, then it must have eol-style
+# POLICY: no svn:executable outside of Tools and svnadmin
+# POLICY: file replacement is not allowed
+verify.py "$REPO" -t "$TXN" || exit 1
+
+# check for forbidden file names
+deny-filenames.sh "$REPO" "$TXN" || exit 1
+
+# check for misplaced mergeinfo
+mergeinfo.sh "$REPO" "$TXN" || exit 1
+
+# check for merge debris
+detect-merge-conflicts.sh "$REPO" "$TXN" || exit 1
+
+# check for newline at end of file
+detect-nonewline-at-eof.sh "$REPO" "$TXN" || exit 1
+
+# check for upper/lowercase filename conflicts on clients
+case-insensitive.py "$REPO" "$TXN" || exit 1
+
+# fix log message.
+log-police.py -t "$TXN" "$REPO" || exit 1
+
+# Nothing else, go ahead.
+exit 0
Property changes on: svnadmin/hooks/pre-commit
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/case-insensitive.py
===================================================================
--- svnadmin/hooks/scripts/case-insensitive.py (rev 0)
+++ svnadmin/hooks/scripts/case-insensitive.py 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,112 @@
+#!/usr/bin/env python
+
+# $MidnightBSD$
+# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/case-insensitive.py
+
+# Licensed under the same terms as Subversion.
+
+# A pre-commit hook to detect case-insensitive filename clashes.
+#
+# What this script does:
+# - Detects new paths that 'clash' with existing, or other new, paths.
+# - Ignores existings paths that already 'clash'
+# - Exits with an error code, and a diagnostic on stderr, if 'clashes'
+# are detected.
+#
+# How it does it:
+# - Get a list of changed paths.
+# - From that list extract the new paths that represent adds or replaces.
+# - For each new path:
+# - Split the path into a directory and a name.
+# - Get the names of all the entries in the version of the directory
+# within the txn.
+# - Compare the canonical new name with each canonical entry name.
+# - If the canonical names match and the pristine names do not match
+# then we have a 'clash'.
+#
+# Notes:
+# - All the paths from the Subversion filesystem bindings are encoded
+# in UTF-8 and the separator is '/' on all OS's.
+# - The canonical form determines what constitutes a 'clash', at present
+# a simple 'lower case' is used. That's probably not identical to the
+# behaviour of Windows or OSX, but it might be good enough.
+# - Hooks get invoked with an empty environment so this script explicitly
+# sets a locale; make sure it is a sensible value.
+# - If used with Apache the 'clash' diagnostic must be ASCII irrespective
+# of the locale, see the 'Force' comment near the end of the script for
+# one way to achieve this.
+#
+# How to call it:
+#
+# On a Unix system put this script in the hooks directory and add this to
+# the pre-commit script:
+#
+# $REPOS/hooks/case-insensitive.py "$REPOS" "$TXN" || exit 1
+#
+
+import sys, locale
+#sys.path.append('/usr/local/subversion/lib/svn-python')
+from svn import repos, fs
+locale.setlocale(locale.LC_ALL, 'C')
+
+def canonicalize(path):
+ return path.decode('utf-8').lower().encode('utf-8')
+
+def get_new_paths(txn_root):
+ new_paths = []
+ for path, change in fs.paths_changed(txn_root).iteritems():
+ if (change.change_kind == fs.path_change_add
+ or change.change_kind == fs.path_change_replace):
+ new_paths.append(path)
+ return new_paths
+
+def split_path(path):
+ slash = path.rindex('/')
+ if (slash == 0):
+ return '/', path[1:]
+ return path[:slash], path[slash+1:]
+
+def join_path(dir, name):
+ if (dir == '/'):
+ return '/' + name
+ return dir + '/' + name
+
+def ensure_names(path, names, txn_root):
+ if (not names.has_key(path)):
+ names[path] = []
+ for name, dirent in fs.dir_entries(txn_root, path).iteritems():
+ names[path].append([canonicalize(name), name])
+
+names = {} # map of: key - path, value - list of two element lists of names
+clashes = {} # map of: key - path, value - map of: key - path, value - dummy
+
+native = locale.getlocale()[1]
+if not native: native = 'ascii'
+repos_handle = repos.open(sys.argv[1].decode(native).encode('utf-8'))
+fs_handle = repos.fs(repos_handle)
+txn_handle = fs.open_txn(fs_handle, sys.argv[2].decode(native).encode('utf-8'))
+txn_root = fs.txn_root(txn_handle)
+
+new_paths = get_new_paths(txn_root)
+for path in new_paths:
+ dir, name = split_path(path)
+ canonical = canonicalize(name)
+ ensure_names(dir, names, txn_root)
+ for name_pair in names[dir]:
+ if (name_pair[0] == canonical and name_pair[1] != name):
+ canonical_path = join_path(dir, canonical)
+ if (not clashes.has_key(canonical_path)):
+ clashes[canonical_path] = {}
+ clashes[canonical_path][join_path(dir, name)] = True
+ clashes[canonical_path][join_path(dir, name_pair[1])] = True
+
+if (clashes):
+ # native = 'ascii' # Force ASCII output for Apache
+ for canonical_path in clashes.iterkeys():
+ sys.stderr.write(u'Clash:'.encode(native))
+ for path in clashes[canonical_path].iterkeys():
+ sys.stderr.write(u' \''.encode(native) +
+ str(path).decode('utf-8').encode(native, 'replace') +
+ u'\''.encode(native))
+ sys.stderr.write(u'\n'.encode(native))
+ sys.exit(1)
Property changes on: svnadmin/hooks/scripts/case-insensitive.py
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/deny-filenames.sh
===================================================================
--- svnadmin/hooks/scripts/deny-filenames.sh (rev 0)
+++ svnadmin/hooks/scripts/deny-filenames.sh 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,38 @@
+#!/bin/sh
+#
+# $MidnightBSD$
+# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/detect-merge-conflicts.sh
+
+# A pre-commit hook to detect forbidden files like core files,
+# patch leftovers, CVS, Subversion or git directories.
+#
+
+REPO=$1
+TXN=$2
+
+SVNLOOK=/usr/local/bin/svnlook
+
+# Check arguments
+if [ -z "$REPO" -o -z "$TXN" ]; then
+ echo "Syntax: $0 path_to_repos txn_id" >&2
+ exit 1
+fi
+
+# We scan through the changed files, looking for forbidden files
+PATH=$($SVNLOOK changed -t $TXN $REPO | /usr/bin/awk '{print $2}')
+for path in $PATH
+do
+ file=`/usr/bin/basename ${path} | /usr/bin/sed -e 's,/$,,'`
+ if echo ${file} | /usr/bin/egrep '^CVS$|^\.svn$|^\.git$' || echo ${file} | \
+ /usr/bin/egrep '\.orig$' || echo ${file} | \
+ /usr/bin/egrep -v '^patch-' | /usr/bin/egrep '\.core$'
+ then
+ echo "Some files in your commit look suspiciously like core files," >&2
+ echo "patch leftovers, CVS, Subversion or git directories. Please" >&2
+ echo "double-check your commit and try committing again." >&2
+ exit 1
+ fi
+done
+
+# No forbidden files detected, let it fly!
+exit 0
Property changes on: svnadmin/hooks/scripts/deny-filenames.sh
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/detect-merge-conflicts.sh
===================================================================
--- svnadmin/hooks/scripts/detect-merge-conflicts.sh (rev 0)
+++ svnadmin/hooks/scripts/detect-merge-conflicts.sh 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,35 @@
+#!/bin/sh
+#
+# $MidnightBSD$
+# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/detect-merge-conflicts.sh
+
+# A pre-commit hook to detect changes that look like forgotten
+# conflict markers. If any additions starting with '>>>>>>>'
+# or '<<<<<<<' are found, the commit is aborted with a nice
+# error message.
+#
+
+REPO=$1
+TXN=$2
+
+SVNLOOK=/usr/local/bin/svnlook
+
+# Check arguments
+if [ -z "$REPO" -o -z "$TXN" ]; then
+ echo "Syntax: $0 path_to_repos txn_id" >&2
+ exit 1
+fi
+
+# We scan through the transaction diff, looking for things that look
+# like conflict markers. If we find one, we abort the commit.
+SUSPICIOUS=$($SVNLOOK diff -t "$TXN" "$REPO" | grep -E '^\+(<{7} \.|>{7} \.)' | wc -l)
+
+if [ $SUSPICIOUS -ne 0 ]; then
+ echo "Some parts of your commit look suspiciously like merge" >&2
+ echo "conflict markers. Please double-check your diff and try" >&2
+ echo "committing again." >&2
+ exit 1
+fi
+
+# No conflict markers detected, let it fly!
+exit 0
Property changes on: svnadmin/hooks/scripts/detect-merge-conflicts.sh
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/detect-nonewline-at-eof.sh
===================================================================
--- svnadmin/hooks/scripts/detect-nonewline-at-eof.sh (rev 0)
+++ svnadmin/hooks/scripts/detect-nonewline-at-eof.sh 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# $MidnightBSD$
+#
+
+# A pre-commit hook to detect files with no newline at end of file
+#
+
+REPO=$1
+TXN=$2
+
+SVNLOOK=/usr/local/bin/svnlook
+
+# Check arguments
+if [ -z "$REPO" -o -z "$TXN" ]; then
+ echo "Syntax: $0 path_to_repos txn_id" >&2
+ exit 1
+fi
+
+# We scan through the transaction diff, looking for
+# 'No newline at end of file' message. If we find one,
+# we abort the commit.
+SUSPICIOUS=$($SVNLOOK diff -t "$TXN" "$REPO" | \
+ grep '^\\ No newline at end of file' | wc -l)
+
+if [ $SUSPICIOUS -ne 0 ]; then
+ echo "Some files in your commit does not have newline at end" >&2
+ echo "of file. Please fix this and try committing again." >&2
+ exit 1
+fi
+
+# No files without newlines at last line detected, let it fly!
+exit 0
Property changes on: svnadmin/hooks/scripts/detect-nonewline-at-eof.sh
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/log-police.py
===================================================================
--- svnadmin/hooks/scripts/log-police.py (rev 0)
+++ svnadmin/hooks/scripts/log-police.py 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,159 @@
+#!/usr/local/bin/python
+
+# $MIdnightBSD$
+# http://svn.collab.net/repos/svn/trunk/tools/hook-scripts/log-police.py
+
+# log-police.py: Ensure that log messages end with a single newline.
+# See usage() function for details, or just run with no arguments.
+
+import os
+import sys
+import getopt
+try:
+ my_getopt = getopt.gnu_getopt
+except AttributeError:
+ my_getopt = getopt.getopt
+
+import svn
+import svn.fs
+import svn.repos
+import svn.core
+
+
+# Pretend we have true booleans on older python versions
+# How old???
+try:
+ True
+except:
+ True = 1
+ False = 0
+
+
+def fix_log_message(log_message):
+ "Return a fixed version of LOG_MESSAGE."
+ s = ""
+ # Pretend leading blanks are adjacent to another
+ lastblank = True
+ for line in log_message.splitlines():
+ line = line.rstrip()
+ # Remove duplicate blank lines
+ if line == "":
+ if lastblank:
+ continue
+ lastblank = True
+ else:
+ lastblank = False
+ # should check last paragraph for known headers.
+ if line == "PR:": continue
+ if line == "Submitted by:": continue
+ if line == "Reviewed by:": continue
+ if line == "Approved by:": continue
+ if line == "Obtained from:": continue
+ if line == "MFC after:": continue
+ if line == "Security:": continue
+ if line == "Sponsored by:": continue
+ s = s + line + "\n"
+ s = s.rstrip() + "\n"
+ return s
+
+
+def fix_txn(fs, txn_name):
+ "Fix up the log message for txn TXN_NAME in FS. See fix_log_message()."
+ txn = svn.fs.svn_fs_open_txn(fs, txn_name)
+ log_message = svn.fs.svn_fs_txn_prop(txn, "svn:log")
+ if log_message is not None:
+ new_message = fix_log_message(log_message)
+ if new_message != log_message:
+ svn.fs.svn_fs_change_txn_prop(txn, "svn:log", new_message)
+
+
+def fix_rev(fs, revnum):
+ "Fix up the log message for revision REVNUM in FS. See fix_log_message()."
+ log_message = svn.fs.svn_fs_revision_prop(fs, revnum, 'svn:log')
+ if log_message is not None:
+ new_message = fix_log_message(log_message)
+ if new_message != log_message:
+ svn.fs.svn_fs_change_rev_prop(fs, revnum, "svn:log", new_message)
+
+
+def usage_and_exit(error_msg=None):
+ """Write usage information and exit. If ERROR_MSG is provide, that
+ error message is printed first (to stderr), the usage info goes to
+ stderr, and the script exits with a non-zero status. Otherwise,
+ usage info goes to stdout and the script exits with a zero status."""
+ import os.path
+ stream = error_msg and sys.stderr or sys.stdout
+ if error_msg:
+ stream.write("ERROR: %s\n\n" % error_msg)
+ stream.write("USAGE: %s [-t TXN_NAME | -r REV_NUM | --all-revs] REPOS\n"
+ % (os.path.basename(sys.argv[0])))
+ stream.write("""
+Ensure that log messages end with exactly one newline and no other
+whitespace characters. Use as a pre-commit hook by passing '-t TXN_NAME';
+fix up a single revision by passing '-r REV_NUM'; fix up all revisions by
+passing '--all-revs'. (When used as a pre-commit hook, may modify the
+svn:log property on the txn.)
+""")
+ sys.exit(error_msg and 1 or 0)
+
+
+def main(ignored_pool, argv):
+ repos_path = None
+ txn_name = None
+ rev_name = None
+ all_revs = False
+
+ try:
+ opts, args = my_getopt(argv[1:], 't:r:h?', ["help", "all-revs"])
+ except:
+ usage_and_exit("problem processing arguments / options.")
+ for opt, value in opts:
+ if opt == '--help' or opt == '-h' or opt == '-?':
+ usage_and_exit()
+ elif opt == '-t':
+ txn_name = value
+ elif opt == '-r':
+ rev_name = value
+ elif opt == '--all-revs':
+ all_revs = True
+ else:
+ usage_and_exit("unknown option '%s'." % opt)
+
+ if txn_name is not None and rev_name is not None:
+ usage_and_exit("cannot pass both -t and -r.")
+ if txn_name is not None and all_revs:
+ usage_and_exit("cannot pass --all-revs with -t.")
+ if rev_name is not None and all_revs:
+ usage_and_exit("cannot pass --all-revs with -r.")
+ if rev_name is None and txn_name is None and not all_revs:
+ usage_and_exit("must provide exactly one of -r, -t, or --all-revs.")
+ if len(args) != 1:
+ usage_and_exit("only one argument allowed (the repository).")
+
+ repos_path = svn.core.svn_path_canonicalize(args[0])
+
+ # A non-bindings version of this could be implemented by calling out
+ # to 'svnlook getlog' and 'svnadmin setlog'. However, using the
+ # bindings results in much simpler code.
+
+ fs = svn.repos.svn_repos_fs(svn.repos.svn_repos_open(repos_path))
+ if txn_name is not None:
+ fix_txn(fs, txn_name)
+ elif rev_name is not None:
+ fix_rev(fs, int(rev_name))
+ elif all_revs:
+ # Do it such that if we're running on a live repository, we'll
+ # catch up even with commits that came in after we started.
+ last_youngest = 0
+ while True:
+ youngest = svn.fs.svn_fs_youngest_rev(fs)
+ if youngest >= last_youngest:
+ for this_rev in range(last_youngest, youngest + 1):
+ fix_rev(fs, this_rev)
+ last_youngest = youngest + 1
+ else:
+ break
+
+
+if __name__ == '__main__':
+ sys.exit(svn.core.run_app(main, sys.argv))
Property changes on: svnadmin/hooks/scripts/log-police.py
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/mergeinfo.sh
===================================================================
--- svnadmin/hooks/scripts/mergeinfo.sh (rev 0)
+++ svnadmin/hooks/scripts/mergeinfo.sh 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,37 @@
+#!/bin/sh
+#
+# $MidnightBSD$
+# http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/detect-merge-conflicts.sh
+
+# A pre-commit hook to check that mergeinfo is only
+# allowed on certain directories.
+#
+
+REPO=$1
+TXN=$2
+
+SVNLOOK=/usr/local/bin/svnlook
+
+# Check arguments
+if [ -z "$REPO" -o -z "$TXN" ]; then
+ echo "Syntax: $0 path_to_repos txn_id" >&2
+ exit 1
+fi
+
+# We scan through the changed directories and looking for mergeinfo
+PATH=$($SVNLOOK dirs-changed -t $TXN $REPO)
+for path in $PATH
+do
+ if $SVNLOOK proplist -t $TXN $REPO ${path} | /usr/bin/grep svn:mergeinfo
+ then
+ if echo ${path} | /usr/bin/egrep -q -v '^branches/[A-Z0-9_]*/$|^tags/[A-Z0-9_]*/$'
+ then
+ echo "It seems that the mergeinfo is at the wrong place." >&2
+ echo "Please double-check your commit and try committing again." >&2
+ exit 1
+ fi
+ fi
+done
+
+# No wrong mergeinfo detected, let it fly!
+exit 0
Property changes on: svnadmin/hooks/scripts/mergeinfo.sh
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
Added: svnadmin/hooks/scripts/verify.py
===================================================================
--- svnadmin/hooks/scripts/verify.py (rev 0)
+++ svnadmin/hooks/scripts/verify.py 2013-09-17 02:04:27 UTC (rev 15451)
@@ -0,0 +1,273 @@
+#!/usr/local/bin/python
+
+# $MidnightBSD$
+# Based on FreeBSD script of the same name.
+# Loosely based on verify-po.py from tools/hook-scripts
+
+import string
+import sys
+import re
+import os.path
+from svn import core, fs, delta, repos
+
+# POLICY: if cvs2svn:cvs-rev must not be set.
+# POLICY: mime-type must be unset, text/*, application/* or image/*
+# POLICY: if a file does has mnbsd:nokeywords, then svn:keywords must not be set
+# POLICY: if a file has binary chars and no mnbsd:notbinary, then pretend its not binary
+# POLICY: if a file is binary, then it must have mime application/* or image/*
+# POLICY: if a file does not have mnbsd:nokeywords, or is binary then svn:keywords must be set
+# POLICY: if svn:keywords is set, $ MidnightBSD $ must be present and condensed.
+# POLICY: no svn:executable outside of Tools and svnadmin
+# POLICY: file replacement is not allowed
+
+
+text_characters = "".join(map(chr, range(32, 127)) + list("\n\r\t\b"))
+_null_trans = string.maketrans("", "")
+okkw = '$' + 'MidnightBSD' + '$'
+badkw = '\$' + 'MidnightBSD' + ':.*\$'
+
+def is_binary(s):
+ if not s: # Empty files are considered text
+ return False
+ if "\0" in s: # NUL char == instant binary classification
+ return True
+
+ # Get the non-text characters (maps a character to itself then
+ # use the 'remove' option to get rid of the text characters.)
+ t = s.translate(_null_trans, text_characters)
+
+ # If more than 30% non-text characters, then
+ # this is considered a binary file
+ # XXX if we include > 128, then reduce fraction
+ if len(t) > len(s) * 0.30:
+ return True
+
+ # No reason to call it binary
+ return False
+
+def mime_ok(mime):
+ "Return True if we accept the mime type"
+ if mime == 'unspecified':
+ return True
+ if mime.startswith('text/'):
+ return True
+ if mime.startswith('application/'):
+ return True
+ if mime.startswith('image/'):
+ return True
+ return False
+
+def check_keywords(s, exempt):
+ "Check if the keyword is ok"
+ r = re.compile(badkw)
+ if r.search(s):
+ return False
+ if exempt:
+ return True
+ if s.find(okkw) != -1:
+ return True
+ return False
+
+# List of directories that we do keyword checking in
+kw_dirs = [
+ ( r'svnadmin/', False ),
+ ( r'branches/', True ),
+ ( r'tags/', True ),
+ ( r'head/', True ),
+]
+
+# How much of path to strip off to get canonical pathname
+kw_prefixes = [
+ r'head/',
+ r'branches/[^/]+/',
+ r'tags/[^/]+/',
+]
+
+kw_exclude = []
+
+# List of directories where svn:executable is allowed
+ex_prefixes = [
+ r'svnadmin',
+ r'Tools',
+]
+
+def kw_checks_exempt(path):
+ # Check to see if we're in a directory that has keyword checking enabled
+ for prefix, stdlayout in kw_dirs:
+ if path.startswith(prefix):
+ if stdlayout:
+ break
+ else:
+ return False
+ else:
+ return True
+ # First, strip off branches/RELENG_8/, tags/RELEASE_8_3_1/ etc to get canonical paths
+ for prefix in kw_prefixes:
+ r = re.compile(prefix)
+ if r.match(path):
+ path = r.sub('', path, 1)
+ # Now, with a canonical path, check for exclusions.
+ for prefix in kw_exclude:
+ r = re.compile(prefix)
+ if r.match(path):
+ return True
+ return False
+
+def kw_checks_executable(path):
+ # First, strip off branches/RELENG_8/, tags/RELEASE_8_3_1/ etc to get canonical paths
+ for prefix in kw_prefixes:
+ r = re.compile(prefix)
+ if r.match(path):
+ path = r.sub('', path, 1)
+ # Now, with a canonical path, check for exclusions.
+ for prefix in ex_prefixes:
+ r = re.compile(prefix)
+ if r.match(path):
+ return True
+ return False
+
+class ChangeReceiver(delta.Editor):
+ def __init__(self, fs_ptr, txn_root, pool):
+ self.fs_ptr = fs_ptr
+ self.txn_root = txn_root
+ self.pool = pool
+ self.failed = 0
+ if fs.is_revision_root(self.txn_root):
+ rev = fs.revision_root_revision(self.txn_root)
+ base_rev = rev - 1
+ else:
+ txn_name = fs.txn_root_name(self.txn_root)
+ txn_ptr = fs.open_txn(self.fs_ptr, txn_name)
+ base_rev = fs.txn_base_revision(txn_ptr)
+ self.base_root = fs.revision_root(fs_ptr, base_rev, pool)
+
+ def do_fail(self, msg):
+ if self.failed == 1:
+ sys.stderr.write("== Additional errors may compound and may not be accurate ==\n")
+ self.failed += 1
+ sys.stderr.write(msg)
+
+ def did_fail(self):
+ return self.failed
+
+ def add_file(self, path, parent_baton,
+ copyfrom_path, copyfrom_revision, file_pool):
+ return [0, path]
+
+ def open_file(self, path, parent_baton, base_revision, file_pool):
+ return [0, path]
+
+ def apply_textdelta(self, file_baton, base_checksum):
+ file_baton[0] = 1
+ # no handler
+ return None
+
+ def close_file(self, file_baton, text_checksum):
+ changed, path = file_baton
+ if not changed:
+ return
+
+ # POLICY: if cvs2svn:cvs-rev must not be set. period.
+ cvsrev = fs.node_prop(self.txn_root, path, 'cvs2svn:cvs-rev')
+ if cvsrev:
+ self.do_fail('Path "%s" needs to have "cvs2svn:cvs-rev" removed with "svn propdel".\n' % path)
+
+ # POLICY: mime-type must be unset, text/*, application/* or image/*
+ mimetype = fs.node_prop(self.txn_root, path, core.SVN_PROP_MIME_TYPE)
+ if not mimetype:
+ mimetype = 'unspecified'
+ if not mime_ok(mimetype):
+ self.do_fail('Path "%s" has an unknown mime type "%s"\n' % (path, mimetype))
+
+ # POLICY: if a file does has mnbsd:nokeywords, then svn:keywords must not be set
+ mnbsd_nokeywords = fs.node_prop(self.txn_root, path, 'mnbsd:nokeywords')
+ keywords = fs.node_prop(self.txn_root, path, core.SVN_PROP_KEYWORDS)
+ if mnbsd_nokeywords and keywords:
+ self.do_fail('Path "%s" has mnbsd:nokeywords AND svn:keywords. Remove one.\n' % path)
+
+ subpool = core.svn_pool_create(self.pool)
+ stream = core.Stream(fs.file_contents(self.txn_root, path, subpool))
+ str_list = []
+ while 1:
+ data = stream.read(core.SVN_STREAM_CHUNK_SIZE)
+ str_list.append(data)
+ if len(data) < core.SVN_STREAM_CHUNK_SIZE:
+ break
+ string = ''.join(str_list)
+
+ # XXX: check for charset in mime type; bypass binary test if charset is present.
+ # POLICY: if a file has binary chars and mnbsd:notbinary, then pretend its not binary
+ binary = is_binary(string)
+ mnbsd_notbinary = fs.node_prop(self.txn_root, path, 'mnbsd:notbinary')
+ if binary and mnbsd_notbinary:
+ binary = False
+
+ # POLICY: if a file is binary, then it must have mime application/* or image/*
+ if binary:
+ if not mimetype.startswith('application/') and not mimetype.startswith('image/'):
+ self.do_fail('Path "%s" contains binary but has svn:mime-type "%s"\n' % (path, mimetype))
+ sys.stderr.write('Try application/* (application/octet-stream) or image/* instead.\n')
+
+ # See which paths don't require the svn:keywords property, or don't need the $ MidnightBSD $ string.
+ kw_exempt = kw_checks_exempt(path)
+
+ # POLICY: if a file does not have mnbsd:nokeywords, and is not binary then svn:keywords must be set
+ if binary:
+ mnbsd_nokeywords = True
+ if not mnbsd_nokeywords and not kw_exempt:
+ kw = r'MidnightBSD=%H'
+ if not keywords:
+ self.do_fail('Path "%s" is missing the svn:keywords property (or an mnbsd:nokeywords override)\n' % path)
+ elif keywords != kw:
+ self.do_fail('Path "%s" should have svn:keywords set to %s\n' % (path, kw))
+
+ # POLICY: if svn:keywords is set, $ MidnightBSD $ must be present and condensed.
+ if keywords and not check_keywords(string, kw_exempt):
+ self.do_fail('Path "%s" does not have a valid %s string (keywords not disabled here)\n' % (path, okkw))
+
+ # POLICY: no svn:executable outside of Tools and svnadmin
+ svn_executable = fs.node_prop(self.txn_root, path, core.SVN_PROP_EXECUTABLE)
+ kw_executable = kw_checks_executable(path)
+ if svn_executable and not kw_executable:
+ self.do_fail('Path "%s" needs to have "svn:executable" removed with "svn propdel".\n' % path)
+
+ # POLICY: file replacement is not allowed
+ for path, change in fs.paths_changed(self.txn_root).iteritems():
+ if (change.change_kind == fs.path_change_replace):
+ self.do_fail('Do not replace a file. This can lose history. Path: "%s"\n' % path)
+
+ # Whew!
+ core.svn_pool_destroy(subpool)
+
+
+def verify(pool, repos_path, mode, rev_or_txn):
+ def authz_cb(root, path, pool):
+ return True
+
+ for line in open(os.path.join(repos_path, 'conf', 'exclude')):
+ ln = line.strip()
+ if not ln.startswith('#') and ln != '':
+ kw_exclude.append(ln)
+ fs_ptr = repos.fs(repos.open(repos_path, pool))
+ if mode == '-r':
+ rev = int(rev_or_txn)
+ txn_root = fs.revision_root(fs_ptr, rev)
+ elif mode == '-t':
+ txn_ptr = fs.open_txn(fs_ptr, rev_or_txn, pool)
+ txn_root = fs.txn_root(txn_ptr, pool)
+ else:
+ sys.exit("arg 2 must be -r or -t")
+ editor = ChangeReceiver(fs_ptr, txn_root, pool)
+ e_ptr, e_baton = delta.make_editor(editor, pool)
+ repos.svn_repos_replay(txn_root, e_ptr, e_baton, pool)
+ fails = editor.did_fail()
+ if fails > 0:
+ if mode == '-r':
+ sys.stderr.write('== Rev %d problem count: %d\n' % (rev, fails))
+ else:
+ sys.stderr.write('== Pre-commit problem count: %d\n' % fails)
+ sys.exit(1)
+
+if __name__ == '__main__':
+ assert len(sys.argv) == 4
+ core.run_app(verify, sys.argv[1], sys.argv[2], sys.argv[3])
Property changes on: svnadmin/hooks/scripts/verify.py
___________________________________________________________________
Added: svn:executable
## -0,0 +1 ##
+*
\ No newline at end of property
More information about the Midnightbsd-cvs
mailing list