Description
Prefetch miss file by aspera, test it on different computer and network. Downloading by prefetch 2.8.2 and ascp 3.5 is fine on my old server.
- sra-tools(2.10.0) and aspera-cli v2.9.1(aspera-connect v2.9.6 also test) were installed by conda.
- prefetch can download sra files in https mode.
- prefetch -t fasp -vv SRR955385
2019-12-11T00:03:51 prefetch.2.10.0: 'tools/ascp/disabled': not found in configuration
2019-12-11T00:03:51 prefetch.2.10.0: Checking 'ascp'
2019-12-11T00:03:51 prefetch.2.10.0: Using 'ascp'
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/opt/aspera/etc/asperaweb_id_dsa.openssh'
2019-12-11T00:03:51 prefetch.2.10.0: '/opt/aspera/etc/asperaweb_id_dsa.openssh': not found
2019-12-11T00:03:51 prefetch.2.10.0: Checking 'ascp'
2019-12-11T00:03:51 prefetch.2.10.0: Using 'ascp'
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/opt/aspera/etc/asperaweb_id_dsa.putty'
2019-12-11T00:03:51 prefetch.2.10.0: '/opt/aspera/etc/asperaweb_id_dsa.putty': not found
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/usr/bin/ascp'
2019-12-11T00:03:51 prefetch.2.10.0: '/usr/bin/ascp': not found
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/usr/bin/ascp'
2019-12-11T00:03:51 prefetch.2.10.0: '/usr/bin/ascp': not found
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/opt/aspera/bin/ascp'
2019-12-11T00:03:51 prefetch.2.10.0: '/opt/aspera/bin/ascp': not found
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/opt/aspera/bin/ascp'
2019-12-11T00:03:51 prefetch.2.10.0: '/opt/aspera/bin/ascp': not found
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/home/mark/.aspera/connect/bin/ascp'
2019-12-11T00:03:51 prefetch.2.10.0: Using '/home/mark/.aspera/connect/bin/ascp'
2019-12-11T00:03:51 prefetch.2.10.0: Checking '/home/mark/.aspera/connect/etc/asperaweb_id_dsa.openssh'
2019-12-11T00:03:51 prefetch.2.10.0: '/home/mark/.aspera/connect/etc/asperaweb_id_dsa.openssh': found
2019-12-11T00:03:52 prefetch.2.10.0: KClientHttpOpen - connected from '192.168.0.102' to trace.ncbi.nlm.nih.gov (130.14.29.113)
2019-12-11T00:03:53 prefetch.2.10.0: KClientHttpOpen - verifying CA cert
2019-12-11T00:03:53 prefetch.2.10.0: KClientHttpOpen - connected from '192.168.0.102' to sra-downloadb.be-md.ncbi.nlm.nih.gov (130.14.250.26)
2019-12-11T00:03:54 prefetch.2.10.0: KClientHttpOpen - verifying CA cert
2019-12-11T00:03:54 prefetch.2.10.0: ########## local(NULL)
2019-12-11T00:03:54 prefetch.2.10.0 err: file not found while copying file - cannot download 'SRR955385' using requested transport
I tried download by ascp directly, it can get the data.
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
Participants
Activity
kwrodarmer commentedon Dec 11, 2019
The SRA no longer resides on ascp compatible media, and while ascp can operate, it no longer offers the benefits it did with the previous storage. Please just use
prefetchwith https mode.indexofire commentedon Dec 14, 2019
So prefetch will cancel ascp support or just https mode recommendation? Was there any info in the Changes.md or release note? Anyway I can still use ascp to download from NCBI host sra-download.ncbi.nlm.nih.gov. I think NCBI will not dismiss ascp media support in the near future, right?
kwrodarmer commentedon Dec 16, 2019
NCBI has shifted to using cloud-style object stores. The Aspera algorithm requires random disk access to perform its function, and random access to object stores gives very poor performance. If Aspera comes out with a server that works well on cloud and object stores, we will support it, but currently there is no Aspera support available to my knowledge.
prefetchis not cancelingascpsupport, as you say, but will remove the option from the command line in the next release because there will not be an Aspera server available for AWS, GCP or NCBI, making the option useless and misleading. It will be added back in if the situation changes.indexofire commentedon Dec 17, 2019
@kwrodarmer Thanks for your clear explaination! I will focus the changes in the future.
hmontenegro commentedon Jun 8, 2020
Some Wiki documentation needs to be updated to reflect the
prefetchchanges, e.g.:HowTo: Access SRA Data
kwrodarmer commentedon Jun 8, 2020
Yes, thank you.