Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Categories

In this Discussion

› › rclone now supports Hubic
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

rclone now supports Hubic

amhoabamhoab Member
in General

rclone is a cli utility that allows you to interact with multiple storage backends. I just saw that they released a new version that supports Hubic. I've done minimal testing, but it appears to work fine so far. I figured this would be of use to many of you here.

Downloads:
http://rclone.org/downloads/

Github:
https://github.com/ncw/rclone

Thanked by 5ATHK tommy Baris inthecloudblog WebProject
«1

Comments

  • MarkTurnerMarkTurner Member

    Whats performance like outside of OVH's network?

  • amhoabamhoab Member
    edited November 2015

    I copied a couple of files, 100MB and 500MB each, from a VPS I have in London. The average throughput was around 50Mbps, although it varied across runs.

    # rclone copy 100mb.test hubic:default/ -v
2015/11/16 07:01:19 Hubic: Got swift credentials (expiry 2015-11-17 04:27:58 +0100 +0100 in 23h26m38.277154845s)
2015/11/16 07:01:19 Hubic Swift container default: Modify window is 1s
2015/11/16 07:01:19 Hubic Swift container default: Building file list
2015/11/16 07:01:20 Hubic Swift container default: Waiting for checks to finish
2015/11/16 07:01:20 Hubic Swift container default: Waiting for transfers to finish
2015/11/16 07:01:35 100mb.test: Copied (new)

Transferred:    104857600 Bytes (6349.81 kByte/s)
Errors:                 0
Checks:                 0
Transferred:            1
Elapsed time:       16.1s

2015/11/16 07:01:35 Go routines at exit 27
  • MarkTurnerMarkTurner Member

    Thats not bad

  • ATHKATHK Member

    Love rclone great rsync like cloud syncing tool.

  • MarkTurnerMarkTurner Member

    I finally got a tutorial for RClone up on Delimiter's wiki. I've been playing with RClone for the past few days. Kudos to @amhoab for the recommendation.

    http://wiki.delimiter.com/display/DEL/ObjSpace:+Sync+with+RClone

    So you can try it properly, there's a free trial of ObjSpace 100GB plan for a month and see how you like it. Its $2.50/month thereafter if you keep it, you can scale up if you need more space.

    http://www.delimiter.com/landingpage/objspace/

    Coupon: 1HD16WH7JS

    Thanked by 1erkin
  • bohdansbohdans Member

    @MarkTurner did the ObjStorage prices drop in the last few days? I am sure 100GB was $10 last time I looked.

  • MarkTurnerMarkTurner Member

    bohdans said: did the ObjStorage prices drop in the last few days? I am sure 100GB was $10 last time I looked.

    100GB is a new entry point, previously the lowest entry point was 500GB for $9.80

    Thanked by 1bohdans
  • inthecloudbloginthecloudblog Member
    edited November 2015

    @amhoab said:
    I copied a couple of files, 100MB and 500MB each, from a VPS I have in London. The average throughput was around 50Mbps, although it varied across runs.

    > # rclone copy 100mb.test hubic:default/ -v
> 2015/11/16 07:01:19 Hubic: Got swift credentials (expiry 2015-11-17 04:27:58 +0100 +0100 in 23h26m38.277154845s)
> 2015/11/16 07:01:19 Hubic Swift container default: Modify window is 1s
> 2015/11/16 07:01:19 Hubic Swift container default: Building file list
> 2015/11/16 07:01:20 Hubic Swift container default: Waiting for checks to finish
> 2015/11/16 07:01:20 Hubic Swift container default: Waiting for transfers to finish
> 2015/11/16 07:01:35 100mb.test: Copied (new)
> 
> Transferred:    104857600 Bytes (6349.81 kByte/s)
> Errors:                 0
> Checks:                 0
> Transferred:            1
> Elapsed time:       16.1s
> 
> 2015/11/16 07:01:35 Go routines at exit 27
>

    Which vps to Hubic?

  • bohdansbohdans Member

    @markturner I like the domain for the ObjStorage

    Thanked by 1MarkTurner
  • tommytommy Member

    from Leaseweb SG to Hubic, I can max out my upload speed

    Transferred:   1000000000 Bytes (8803.78 kByte/s)
Errors:                 0
Checks:                 0
Transferred:            1
Elapsed time:     1m50.9s

    test upload 954 M

    Thanked by 1vimalware
  • TinyTunnel_TomTinyTunnel_Tom Member

    OVH>Online.net is horrid. 40Gbit of connectivity on somedays 2 of the links are at 97% the other 2 are at 92%

  • rmlhhdrmlhhd Member
    edited November 2015

    @MarkTurner said:

    So you can try it properly, there's a free trial of ObjSpace 100GB plan for a month and see how you like it. Its $2.50/month thereafter if you keep it, you can scale up if you need more space
    Coupon: 1HD16WH7JS

    Shame you only have a US location for object storage

  • MarkTurnerMarkTurner Member

    rmlhhd said: Shame you only have a US location for object storage

    If there is demand, then we could deploy in any of our EU sites, object storage is sufficiently profitable to operate in EU and Asia.

  • NihimNihim Member
    edited November 2015

    Can someone check how fast files are deleted? If I remember when I got my hubic acc people were saying files too a long as time to get removed.

    Also using some hubic api at that time, any file uploaded had to be copied first to some special partition or dir of the api (which was quite a waste of I/O).

    Does that still happen with the above method?

    ps: obviously I never did manage to test for myself even though my hubic 10T acc is active and paid since their offer back then :/

  • mosan7763mosan7763 Member
    edited November 2015

    any guide to set it up on debian? when following the linux guide i just get stuck at "waiting for code..".

    the webpage reports: "Failed! No code found." (I access it over ssh tunnel)...

  • CFarenceCFarence Member

    @MarkTurner said:
    Whats performance like outside of OVH's network?

    I just transferred 800MB from Delimiter to Hubic and got 5056.04 kByte/s.

    Thanked by 2MarkTurner vimalware
  • NihimNihim Member

    question: rclone site says that you need to open a browser so it gets the hubic token.

    What if you are running cli?

  • BochiBochi Member

    @Nihim said:
    question: rclone site says that you need to open a browser so it gets the hubic token.

    What if you are running cli?

    No problem at all, just tested it myself: Only thing you need to do is forwarding a port, then you can go through the usual authentication process of hubiC.

    Thanked by 1Nihim
  • NihimNihim Member

    @Bochi said:
    No problem at all, just tested it myself: Only thing you need to do is forwarding a port, then you can go through the usual authentication process of hubiC.

    Thanks. As I said I haven't tried it myself so not quite sure what you did.
    If you feel like it you could drop a few lines of noob-proof steps.

  • mosan7763mosan7763 Member

    somehow the upload speed from online.net with rclone is only 7mbit. I usually got 150-200mbit with hubiCs own app and about 100mbit with hubicfuse. strange.

  • BochiBochi Member
    edited November 2015

    @Nihim said:
    If you feel like it you could drop a few lines of noob-proof steps.

    Sure! Just answered a question regarding this by a private message...

    For example with Putty (ssh on linux should achieve the same):
    It is as simple as creating a tunnel to "127.0.0.1:53682", then you can access the remote path on your local source port and go through the normal App-Login mechanism of hubiC.
    Client ID and secret will then be stored in your local config file afterwards and be used to access your storage.

    /EDIT/
    But as far as I can judge that after this short time it seems as if I get slower uploads as for example with duply or their original client.

    Thanked by 1ATHK
  • KupolKupol Member

    @mosan7763

    online.net's links to OVH are overloaded most of the time. This could cause slow speed to OVH from online.net.

  • amhoabamhoab Member

    @Nihim said:
    question: rclone site says that you need to open a browser so it gets the hubic token.

    What if you are running cli?

    I just opened another shell with tmux and used lynx to open that URL. rclone was able to detect that, and updated the config automatically.

  • amhoabamhoab Member

    @inthecloudblog said:
    Which vps to Hubic?

    This was from a We Love Servers VM in London.

    Thanked by 1inthecloudblog
  • mosan7763mosan7763 Member
    edited November 2015

    only problem i have with rclone is that the transfer speed is always ~800kbyte/s per file. it doesn't matter how many (1 or 20) it's always ~800kbyte/s.

  • MarkTurnerMarkTurner Member

    mosan7763 said: only problem i have with rclone is that the transfer speed is always

    Rclone to what?

  • mosan7763mosan7763 Member

    hubiC

  • MarkTurnerMarkTurner Member

    @mosan7763 - Thats more likely to be hubic/network than Rclone. Quite a few people complain about performance of hubic even with the hubic client.

    I get almost wirespeed from my test server to Delimiter ObjSpace using Rclone so the software seems to be perfect.

  • mosan7763mosan7763 Member

    a simple trick is to just queue more files.

  • NeoonNeoon Community Contributor, Veteran

    Hubic is limited to 1MB/s, so 800kb/s is pretty close.

Sign In or Register to comment.