xcookie.main module

This is a Python script to apply the xcookie template to either create a new repo or update an existing one with the latest standards.

Todo

Port logic from ~/misc/make_new_python_package_repo.sh

CommandLine

~/code/xcookie/xcookie/main.py

python -m xcookie.main

ExampleUsage:

# Update my repos python -m xcookie.main –repodir=$HOME/code/pyflann_ibeis –tags=”erotemic,github,binpy”

python -m xcookie.main –repodir=$HOME/code/whodat –tags=”kitware,gitlab,purepy,cv2,gdal” python -m xcookie.main –repodir=$HOME/code/whatdat –tags=”kitware,gitlab,purepy,cv2,gdal” python -m xcookie.main –repodir=$HOME/code/whendat –tags=”kitware,gitlab,purepy,cv2,gdal” python -m xcookie.main –repodir=$HOME/code/whydat –tags=”kitware,gitlab,purepy,cv2,gdal” python -m xcookie.main –repodir=$HOME/code/howdat –tags=”kitware,gitlab,purepy,cv2,gdal”

python -m xcookie.main –repodir=$HOME/code/kwconf –tags=”kitware,gitlab,erotemic,github,purepy”

# Create this repo python -m xcookie.main –repo_name=xcookie –repodir=$HOME/code/xcookie –tags=”erotemic,github,purepy”

# Create a new python repo python -m xcookie.main –repo_name=cookiecutter_purepy –repodir=$HOME/code/cookiecutter_purepy –tags=”github,purepy”

# Create a new binary repo python -m xcookie.main –repo_name=cookiecutter_binpy –repodir=$HOME/code/cookiecutter_binpy –tags=”github,binpy,gdal”

# Create a new binary gitlab kitware repo python -m xcookie.main –repo_name=kwimage_ext –repodir=$HOME/code/kwimage_ext –tags=”kitware,gitlab,binpy” python -m xcookie.main –repo_name=balanced_sampler –repodir=$HOME/code/balanced_sampler –tags=”kitware,gitlab,binpy”

python -m xcookie.main –repo_name=kwcoco_dataloader –repodir=$HOME/code/kwcoco_dataloader –tags=”kitware,gitlab,purepy,gdal,cv2”

# Create a new binary github repo python -m xcookie.main –repodir=$HOME/code/networkx_algo_common_subtree –tags=”github,erotemic,binpy”

# Create a new purepy github repo python -m xcookie.main –repodir=$HOME/code/googledoc –tags=”github,erotemic,purepy”

python -m xcookie.main –repodir=$HOME/code/networkx_algo_common_subtree_cython –tags=”github,erotemic,binpy”

python -m xcookie.main –repo_name=delayed_image –repodir=$HOME/code/delayed_image –tags=”kitware,gitlab,purepy,cv2,gdal”

HOST=https://gitlab.kitware.com export PRIVATE_GITLAB_TOKEN=$(git_token_for “$HOST”) python -m xcookie.main –repo_name=kwutil –repodir=$HOME/code/kwutil –tags=”kitware,gitlab,purepy”

python -m xcookie.main –repo_name=geowatch –repodir=$HOME/code/geowatch –tags=”kitware,gitlab,purepy,cv2,gdal”

python -m xcookie.main –repo_name=stdx –repodir=$HOME/code/stdx –tags=”github,purepy,erotemic”

python -m xcookie.main –repo_name=ustd –repodir=$HOME/code/ustd –tags=”github,purepy,erotemic”

load_secrets export PRIVATE_GITLAB_TOKEN=$(git_token_for “https://gitlab.kitware.com”) python -m xcookie.main –repo_name=simple_dvc –repodir=$HOME/code/simple_dvc –tags=”gitlab,kitware,purepy,erotemic”

python -m xcookie.main –repo_name=audio_restore –repodir=$HOME/code/audio_restore –tags=”github,erotemic,purepy” –use_pyproject_requirements=True –use_setup_py=False

exception xcookie.main.SkipFile

Bases: Exception

class xcookie.main.XCookieConfig(*args: Any, **kwargs: Any)

Bases: DataConfig

The XCookie CLI

Valid options: []

Parameters:

• *args – positional arguments for this data config

• **kwargs – keyword arguments for this data config

_load_pyproject_config()

_load_xcookie_pyproject_settings()

_infer_xcookie_settings_from_pyproject(disk_config)

Helper to populate the xcookie main settings from more standard pyproject schemas.

confirm(msg, default=True)

Parameters:

• msg (str) – display to the user

• default (bool) – default value if non-interactive

Return type:

bool

prompt(msg, choices, default=True)

Parameters:

• msg (str) – display to the user

• default (bool) – default value if non-interactive

Return type:

bool

classmethod load_from_cli_and_pyproject(argv=False, **kwargs)

classmethod main(argv=False, **kwargs)

Main entry point

Example

repodir = ub.Path.appdir(‘pypkg/demo/my_new_repo’) import sys, ubelt sys.path.append(ubelt.expandpath(‘~/code/xcookie’)) from xcookie.main import * # NOQA kwargs = {

‘repodir’: repodir,

} argv = 0

default = {'author': <Value(None)>, 'author_email': <Value(None)>, 'autostage': <Value(False)>, 'ci_blocklist': <Value([])>, 'ci_cpython_versions': <Value('auto')>, 'ci_extras': <Value(None)>, 'ci_gpg_secret_transport': <Value('encrypted_repo')>, 'ci_pypi_live_password_varname': <Value('TWINE_PASSWORD')>, 'ci_pypi_test_password_varname': <Value('TEST_TWINE_PASSWORD')>, 'ci_pypi_trusted_publishing': <Value(False)>, 'ci_pypy_versions': <Value('auto')>, 'ci_versions_full_loose': <Value('*')>, 'ci_versions_full_strict': <Value('main')>, 'ci_versions_minimal_loose': <Value('main')>, 'ci_versions_minimal_strict': <Value('min')>, 'defaultbranch': <Value('main')>, 'deploy': <Value(True)>, 'deploy_artifacts': <Value(True)>, 'deploy_gitlab': <Value(True)>, 'deploy_pypi': <Value(True)>, 'deploy_tags': <Value(True)>, 'description': <Value(None)>, 'dev_status': <Value('planning')>, 'enable_gpg': <Value(True)>, 'init_new_remotes': <Value(True)>, 'interactive': <Value(True)>, 'is_new': <Value('auto')>, 'license': <Value(None)>, 'linter': <Value(True)>, 'main_python': <Value('max')>, 'max_python': <Value(None)>, 'min_python': <Value('3.7')>, 'mod_name': <Value(None)>, 'only_generate': <Value(None)>, 'os': <Value('all')>, 'pkg_name': <Value(None)>, 'refresh_docs': <Value('auto')>, 'regen': <Value(None)>, 'rel_mod_parent_dpath': <Value('.')>, 'remote_group': <Value(None)>, 'remote_host': <Value(None)>, 'render_doc_images': <Value(False)>, 'repo_name': <Value(None)>, 'repodir': <Value('.')>, 'rotate_secrets': <Value('auto')>, 'skip_autogen': <Value(None)>, 'supported_python_versions': <Value('auto')>, 'tags': <Value('auto')>, 'test_command': <Value('auto')>, 'test_env': <Value(None)>, 'test_variants': <Value(['full-loose', 'full-strict', 'minimal-loose', 'minimal-strict'])>, 'typed': <Value(None)>, 'url': <Value(None)>, 'use_pyproject_requirements': <Value(False)>, 'use_setup_py': <Value(True)>, 'use_uv': <Value('auto')>, 'use_vcs': <Value('auto')>, 'version': <Value(None)>, 'visibility': <Value('public')>, 'xdoctest_style': <Value('google')>, 'yes': <Value(False)>}

normalize()

class xcookie.main.TemplateApplier(config)

Bases: object

The primary xcookie autogeneration class.

Note

this does not write any files unless you call setup (which just writes to a temporary directory) or apply (which can destructively clobber things).

apply()

Does the actual modification of the target repo.

Has special logic to handle building new respos versus updating repos.

autostage()

property has_git

property rel_mod_dpath

property mod_dpath

property mod_name

property pkg_name

property pkg_fname_prefix

_build_template_registry()

Take stock of the files in the template repo and ensure they all have appropriate properties.

property tags

_project_classifiers()

_presetup()

This logic used to live in setup, but it doesn’t have external side effects, so it would be good if we were able have these fields populated on initialization for tests.

setup()

Finalizes a few variables and writes the “clean” template to the staging directory.

copy_staged_files()

vcs_checks()

_stage_file(info)

Write a single file to the staging directory based on its template info.

Parameters:

info (dict) – a template dictionary that defines how to construct a file

Returns:

enriched information.

A side effect of this function is writing the data to temporary storage

Return type:

dict

Example

>>> from xcookie.main import *  # NOQA
>>> dpath = ub.Path.appdir('xcookie/tests/test-stage').delete().ensuredir()
>>> kwargs = {
>>>     'repodir': dpath / 'testrepo',
>>>     'tags': ['gitlab', 'kitware', 'purepy', 'cv2'],
>>>     'rotate_secrets': False,
>>>     'is_new': False,
>>>     'interactive': False,
>>> }
>>> config = XCookieConfig.cli(argv=0, data=kwargs)
>>> #config.__post_init__()
>>> print('config = {}'.format(ub.urepr(dict(config), nl=1)))
>>> self = TemplateApplier(config)
>>> self._build_template_registry()
>>> info = [d for d in self.template_infos if d['fname'] == 'setup.py'][0]
>>> info = [d for d in self.template_infos if d['fname'] == '.gitlab-ci.yml'][0]
>>> self._stage_file(info)

_apply_xcookie_directives(stage_fpath)

stage_files()

gather_tasks()

build_requirements_txt()

refresh_docs()

rotate_secrets()

print_help_tips()

build_readthedocs()

Returns:

templated code

Return type:

str

build_setup()

Returns:

templated code

Return type:

str

build_pyproject()

Returns:

templated code

Return type:

str

format_code(text, filename='snippet.py')

Format Python code using the project’s pyproject.toml ruff settings.

Reads ruff configuration from [tool.ruff] and [tool.ruff.format] sections of the project’s pyproject.toml and uses those as defaults for formatting.

Parameters:

• text (str) – Python code to format

• filename (str) – Virtual filename for the formatter (default: ‘snippet.py’)

Returns:

Formatted code

Return type:

str

_setup_pip_commands()

build_github_actions()

build_github_actions_tests()

build_github_actions_release()

build_gitlab_ci()

build_manifest_in()

build_run_linter()

build_gitlab_rules()

build_readme()

build_docs_index()

build_docs_conf()

build_docs_requirements()

build_optional_requirements()

build_runtime_requirements()

build_tests_requirements()

_build_special_requirements(variant, version_defaults, header_lines)

Example

>>> from xcookie.main import *  # NOQA
>>> dpath = ub.Path.appdir('xcookie/tests/test-stage').delete().ensuredir()
>>> kwargs = {
>>>     'repodir': dpath / 'testrepo',
>>>     'tags': ['gitlab', 'kitware', 'purepy', 'cv2'],
>>>     'rotate_secrets': False,
>>>     'is_new': False,
>>>     'min_python': '3.9',
>>>     'max_python': '3.12',
>>>     'interactive': False,
>>> }
>>> config = XCookieConfig.cli(argv=0, data=kwargs)
>>> print('config = {}'.format(ub.urepr(dict(config), nl=1)))
>>> self = TemplateApplier(config)
>>> print(chr(10) + 'headless.txt')
>>> print(self.build_cv2_headless_requirements_txt())
>>> print(chr(10) + 'gdal.txt')
>>> print(self.build_gdal_requirements_txt())

_build_cv2_requirements(variant)

build_cv2_headless_requirements_txt()

build_cv2_graphics_requirements_txt()

build_gdal_requirements_txt()

build_run_doctests()

lut(info)

Hacked builders when template_info source is dynamic, but there is no corresponding explicit function.

Returns:

templated code

Return type:

str

_docs_quickstart()

Todo

• [ ] Auto-edit the index.py to include the magic reference to __init__

• [ ] If this is a utility library, populate the “usefulness section”

• [ ] Try and find out how to auto-expand the toc tree

• [ ] Auto-run the sphinx-apidoc for the user

REPO_NAME=xcookie REPO_DPATH=$HOME/code/$REPO_NAME AUTHOR=$(git config –global user.name) cd $REPO_DPATH/docs sphinx-quickstart -q –sep

–project=”$REPO_NAME” –author=”$AUTHOR” –ext-autodoc –ext-viewcode –ext-intersphinx –ext-todo –extensions=sphinx.ext.autodoc,sphinx.ext.viewcode,sphinx.ext.napoleon,sphinx.ext.intersphinx,sphinx.ext.todo,sphinx.ext.autosummary “$REPO_DPATH/docs”

# THEN NEED TO: REPO_NAME=kwarray REPO_DPATH=$HOME/code/$REPO_NAME MOD_DPATH=$REPO_DPATH/$REPO_NAME sphinx-apidoc -f -o “$REPO_DPATH/docs/source” “$MOD_DPATH” –separate cd “$REPO_DPATH/docs” make html

The user will need to enable the repo on their readthedocs account: https://readthedocs.org/dashboard/import/manual/?

To enable the read-the-docs go to https://readthedocs.org/dashboard/ and login

Make sure you have a .readthedocs.yml file

Click import project: (for github you can select, but gitlab you need to import manually)

Set the Repository NAME: $REPO_NAME Set the Repository URL: $REPO_URL

For gitlab you also need to setup an integrations and add gitlab incoming webhook Then go to $REPO_URL/hooks and add the URL

Will also need to activate the main branch:

https://readthedocs.org/projects/xcookie/versions/

xcookie.main.main()

xcookie.main._parse_remote_url(url)

xcookie.main.find_git_root(dpath)

class xcookie.main.GitURL(data)

Bases: str

Represents a url to a git repo and can parse info about / modify the protocol

References

https://git-scm.com/docs/git-clone#_git_urls

TODO: can use git-well as a helper here.

CommandLine

xdoctest -m /home/joncrall/code/xcookie/xcookie/main.py GitURL
xdoctest -m xcookie.main GitURL

Example

>>> urls = [
>>>     GitURL('https://foo.bar/user/repo.git'),
>>>     GitURL('ssh://foo.bar/user/repo.git'),
>>>     GitURL('ssh://git@foo.bar/user/repo.git'),
>>>     GitURL('git@foo.bar:group/repo.git'),
>>>     GitURL('host:path/to/my/repo/.git'),
>>> ]
>>> for url in urls:
>>>     print('---')
>>>     print(f'url = {url}')
>>>     print(ub.urepr(url.info))
>>>     print('As git   : ' + url.to_git())
>>>     print('As ssh   : ' + url.to_ssh())
>>>     print('As https : ' + url.to_https())

_parse()

property info

to_git()

to_ssh()

to_https()