Skip to content

Alma documentation cleanup#1935

Merged
bsipocz merged 14 commits into
astropy:mainfrom
tinuademargaret:alma-documentation-cleanup
Mar 19, 2022
Merged

Alma documentation cleanup#1935
bsipocz merged 14 commits into
astropy:mainfrom
tinuademargaret:alma-documentation-cleanup

Conversation

@tinuademargaret

Copy link
Copy Markdown
Contributor

No description provided.

@astropy-bot astropy-bot Bot added the alma label Jan 12, 2021
@codecov

codecov Bot commented Jan 12, 2021

Copy link
Copy Markdown

Codecov Report

Merging #1935 (9819ea9) into main (e274a2b) will not change coverage.
The diff coverage is n/a.

❗ Current head 9819ea9 differs from pull request most recent head 5aefe0c. Consider uploading reports for the commit 5aefe0c to get more accurate results

@@           Coverage Diff           @@
##             main    #1935   +/-   ##
=======================================
  Coverage   62.97%   62.97%           
=======================================
  Files         131      131           
  Lines       17073    17073           
=======================================
  Hits        10752    10752           
  Misses       6321     6321           

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@tinuademargaret tinuademargaret added this to the v0.4.2 milestone Jan 19, 2021
@bsipocz bsipocz modified the milestones: v0.4.2, v0.4.3 May 15, 2021
@bsipocz bsipocz removed this from the v0.4.3 milestone Jul 7, 2021
@tinuademargaret tinuademargaret force-pushed the alma-documentation-cleanup branch from b363e19 to bb62694 Compare August 9, 2021 00:17
@tinuademargaret tinuademargaret added this to the v0.4.4 milestone Aug 9, 2021

@bsipocz bsipocz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are unrelated failures, ideally those should be resolved before merging this, to ensure the cleaned up docs indeed passes tests.

Comment thread CHANGES.rst
Comment thread docs/alma/alma.rst
Comment thread docs/alma/alma.rst Outdated
@bsipocz bsipocz removed this from the v0.4.4 milestone Nov 13, 2021
@bsipocz

bsipocz commented Nov 24, 2021

Copy link
Copy Markdown
Member

@keflavich - could you take over this PR to enable doctesting on the alma docs?

@keflavich

Copy link
Copy Markdown
Contributor

I'm working on it, but I get this weird error:

astropy.utils.exceptions.AstropyWarning: leap-second auto-update failed due to the following exception: DeprecationWarning('`np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.\nDeprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations')
/Users/adam/repos/astroquery/docs/alma/alma.rst:25: UnexpectedException

that is unrelated. This feels like something caused by an out-of-date astropy, but I'm testing w/5.1.dev184+gbf35e6f60b

@keflavich keflavich force-pushed the alma-documentation-cleanup branch from d5ba758 to 6fcc13d Compare November 24, 2021 18:47
@bsipocz

bsipocz commented Nov 24, 2021

Copy link
Copy Markdown
Member

There are two unrelated commits picked up. I'll look into how we could filter out that leap second warning, and also ping @eteq as this leap second thing comes up regularly, so there may be a solution for it that I cannot remember from the top of my head.

@bsipocz

bsipocz commented Nov 25, 2021

Copy link
Copy Markdown
Member

@keflavich - That leap-second warning is already on the ignore list, so I'm not sure why it's causing issues locally. Did you see it before or after you made the rebase?

@keflavich keflavich force-pushed the alma-documentation-cleanup branch 2 times, most recently from 62776ea to 70e53e8 Compare November 25, 2021 15:17
@keflavich

Copy link
Copy Markdown
Contributor

I fixed the rebase - some weird things happened that I didn't understand. I had to rebase several times, even after updating the remote.

I had to add a different warning catch for this warning:

DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
/Users/adam/repos/astroquery/docs/alma/alma.rst:25: UnexpectedException

but I don't think that is right, something local must be broken again.

@keflavich keflavich force-pushed the alma-documentation-cleanup branch from 70e53e8 to dc408e7 Compare November 25, 2021 15:39
Comment thread docs/alma/alma.rst Outdated
Comment thread setup.cfg Outdated
# This is a temporary measure, all of these should be fixed:
ignore::pytest.PytestUnraisableExceptionWarning
ignore::numpy.VisibleDeprecationWarning
ignore::DeprecationWarning

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't do this as we need to catch deprecations that we trigger with astroquery code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ach, yes, this was a mistake: for reasons I don't understand, I have to have this for tests to pass locally, but I didn't mean to commit it.

@bsipocz bsipocz added this to the v0.4.5 milestone Nov 26, 2021
@keflavich

Copy link
Copy Markdown
Contributor

I'm getting a ton of exceptions like these, but they're totally unrelated to this PR. I think they're slowing down/screwing with the testing.

Exception ignored in: <ssl.SSLSocket fd=21, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.13.167.149', 39336), raddr=('192.33.115.31', 443)>
Traceback (most recent call last):
  File "/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 59, in __exit__
    del self.unraisable
ResourceWarning: unclosed <ssl.SSLSocket fd=21, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.13.167.149', 39336), raddr=('192.33.115.31', 443)>

@keflavich

Copy link
Copy Markdown
Contributor

I also see tons of these:

Exception ignored in: <ssl.SSLSocket fd=21, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.0.0.44', 57697), raddr=('192.33.115.31', 443)>
ResourceWarning: unclosed <ssl.SSLSocket fd=21, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.0.0.44', 57697), raddr=('192.33.115.31', 443)>

but... why? We shouldn't have unclosed sockets, right? We're just doing a few requests? Maybe this is within requests?

@bsipocz

bsipocz commented Nov 27, 2021

Copy link
Copy Markdown
Member

I recall seeing those socket ones occasionally, but don't think it's related to the content of this PR, and suspect they are upstream either pytest or request. Do you recall seeing any from the python terminal, or they only pop up while you run the tests?

@keflavich

Copy link
Copy Markdown
Contributor

I don't see them on the command line, just during tests.

The doctests are super slow to run; I think they're downloading a lot of data. That might be necessary, but it certainly has made testing inconvenient.

@bsipocz

bsipocz commented Nov 29, 2021

Copy link
Copy Markdown
Member

@keflavich - I also see that they run a very long time, though I don't have many files permanently downloaded. It would be good if the problematic ones would be rephrased, or at least skipped. I would prefer the rewrite as I think it's more useful for users if they can copy-paste examples that actually run and finishes within a reasonable time, they can then rescale it for their own use case.

@keflavich

Copy link
Copy Markdown
Contributor

I think we should go ahead and merge this, but we might need a fresh PR to clean up some more details on another (future) review.

Has the readthedocs build been pending since christmas?

@pllim pllim closed this Jan 26, 2022
@pllim pllim reopened this Jan 26, 2022
@pllim

pllim commented Jan 26, 2022

Copy link
Copy Markdown
Member

There is a problem with your build: https://readthedocs.org/projects/astroquery/builds/15892183/

@bsipocz

bsipocz commented Jan 26, 2022

Copy link
Copy Markdown
Member

@keflavich - does this build locally? As I recall it was hung up for me when I tried it before the holidays. We cannot merge if it hangs as it would block the rest of the package.

@keflavich

Copy link
Copy Markdown
Contributor

I didn't realize there was a real build issue here. I'll investigate when I get back to this.

@bsipocz

bsipocz commented Mar 19, 2022

Copy link
Copy Markdown
Member

This PR should also remove or refactor the test test_doc_example in the remote tests file, too.

@bsipocz bsipocz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to rebase this and will squash a few commits to address my comments. Hopefully that will trigger RDT and it won't timeout on us any longer.

Comment thread docs/alma/alma.rst Outdated
>>> print(len(m83_data))
830
>>> m83_data.colnames
>>> m83_data.colnames # doctest: +SKIP

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be IGNORE_OUTPUT rather than skip

Suggested change
>>> m83_data.colnames # doctest: +SKIP
>>> m83_data.colnames # doctest: +IGNORE_OUTPUT

Comment thread docs/alma/alma.rst Outdated
>>> from astroquery.alma import Alma
>>> m83_data = Alma.query_object('M83')
>>> uids = np.unique(m83_data['member_ous_uid'])
>>> print(uids) # doctest: +IGNORE_OUTPUT

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these expected to change? If not, then we shall not ignore

Comment thread docs/alma/alma.rst Outdated
Comment on lines +326 to +329
.. doctest-remote-data::
>>> myAlma = Alma() # doctest: +SKIP
>>> myAlma.cache_location = '/big/external/drive/' # doctest: +SKIP
>>> myAlma.download_files(link_list, cache=True) # doctest: +SKIP

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as all is skipped here, maybe leave it without remote data

@bsipocz bsipocz force-pushed the alma-documentation-cleanup branch from 57e63c8 to 6686a03 Compare March 19, 2022 04:24
@bsipocz bsipocz modified the milestones: v4, v0.4.6 Mar 19, 2022

@bsipocz bsipocz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing locally, so we're good to go if the RTD rendering works.

@bsipocz bsipocz force-pushed the alma-documentation-cleanup branch from e40cb4c to 9819ea9 Compare March 19, 2022 04:37
@bsipocz

bsipocz commented Mar 19, 2022

Copy link
Copy Markdown
Member

Hmm, tests pass, but sphinx is still not happy, atm it's an IndexError: https://readthedocs.org/projects/astroquery/builds/16411376/

reading sources... [  0%] alma/alma
/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/astroquery/ipac/irsa/sha/__init__.py:14: UserWarning: Experimental: SHA has not yet been refactored to have its API match the rest of astroquery.
  warnings.warn("Experimental: SHA has not yet been refactored to have its "
/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/astroquery/alfalfa/__init__.py:15: UserWarning: Experimental: ALFALFA has not yet been refactored to have its API match the rest of astroquery.
  warnings.warn("Experimental: ALFALFA has not yet been refactored to have "
WARNING: Astrometry.net API key not found in configuration file [astroquery.astrometry_net.core]
WARNING: You need to manually edit the configuration file and add it [astroquery.astrometry_net.core]
WARNING: You may also register it for this session with AstrometryNet.key = 'XXXXXXXX' [astroquery.astrometry_net.core]
/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/astroquery/ogle/__init__.py:40: UserWarning: Experimental: OGLE has not yet been refactored to have its API match the rest of astroquery.
  warnings.warn("Experimental: OGLE has not yet been refactored to have its "
WARNING: AstropyDeprecationWarning: the module relies on an unmaintained library and isconsidered deprecated until completely refactored or upstreamis stablised. [astroquery.vamdc.core]
/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/astroquery/vamdc/core.py:116: UserWarning: vamdclib could not be imported; the vamdc astroquery module will not work
  warnings.warn("vamdclib could not be imported; the vamdc astroquery module "
WARNING: AstropyDeprecationWarning: the module relies on an unmaintained library and isconsidered deprecated until completely refactored or upstreamis stablised. [astroquery.vamdc.core]
/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/astroquery/fermi/__init__.py:37: UserWarning: Experimental: Fermi-LAT has not yet been refactored to have its API match the rest of astroquery.
  warnings.warn("Experimental: Fermi-LAT has not yet been refactored to have "

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/cmd/build.py", line 284, in build_main
    app.build(args.force_all, filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/application.py", line 337, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 294, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 308, in build
    updated_docnames = set(self.read())
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 415, in read
    self._read_serial(docnames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 436, in _read_serial
    self.read_doc(docname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 476, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/io.py", line 189, in read_doc
    pub.publish()
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/io.py", line 109, in read
    self.parse()
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/sphinx/parsers.py", line 101, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 3008, in text
    self.section(title.lstrip(), source, style, lineno + 1, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
    return self.run_directive(
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/pytest_doctestplus/sphinx/doctestplus.py", line 23, in run
    if re.match('win32', self.content[0]):
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 1147, in __getitem__
    return self.data[i]
IndexError: list index out of range

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/astroquery/envs/1935/lib/python3.8/site-packages/docutils/statemachine.py", line 1147, in __getitem__
    return self.data[i]
IndexError: list index out of range
The full traceback has been saved in /tmp/sphinx-err-ma4nbt31.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

@bsipocz

bsipocz commented Mar 19, 2022

Copy link
Copy Markdown
Member

Apparently, two directives cannot be used at once...

@bsipocz bsipocz merged commit 60db44d into astropy:main Mar 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants