https://linen.dev logo
Powered by
Unable to deploy if git present
# help
h
Hey. for some reasons im unable to deloy repo with initialized git repo and at least one commit which was successfully deployed previous without git.
Copy code
git init .
git add .
git commit -am "Initial"
cargo shuttle deploy ---> Deployment crashed. 

cargo shuttle logs MY-LOG --->

2022-10-31T18:01:24.360919797Z  INFO Entering queued state


2022-10-31T18:01:24.363976611Z  INFO Entering building state
2022-10-31T18:01:24.383713394Z  INFO shuttle_deployer::deployment::queue: Extracting received data
2022-10-31T18:01:24.440633930Z  INFO shuttle_deployer::deployment::queue: Building deployment

2022-10-31T18:01:24.447039691Z  INFO Entering crashed state
2022-10-31T18:01:24.465544314Z ERROR {error="Build error: failed to read `/opt/shuttle/shuttle-builds/rutexws/Cargo.toml`"} shuttle_deployer::deployment::queue: service build encountered an error
if im deleting .git folder from project, i am able to deploy project again.
k
Currently we use 
cargo package
to package users crates before we send them to the build server, and this comes with some rules (ref ). My guess is that this particular flow causes the files to be ignored by 
cargo package
, but I'm not sure why yet. I'll take a closer look in the morning and see if I can reproduce this.
h
I see... I will be happy to help you out with this issue if you need more info.
s
What is the output of 
cargo package --list
with and without the .git folder?
h
Without git
Copy code
warning: manifest has no description, license or license-file.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
Cargo.toml
Cargo.toml.orig
public\index.html
rrutexws.code-workspace
src\StellarObjectImpl.rs
src\WebHandlerImpl.rs
src\WebsocketImpl.rs
src\lib.rs
src\test.rs
After git init .
Copy code
warning: manifest has no description, license or license-file.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
error: 8 files in the working directory contain changes that were not yet committed into git:

Cargo.toml
public\index.html
rrutexws.code-workspace
src\lib.rs
src\StellarObjectImpl.rs
src\test.rs
src\WebHandlerImpl.rs
src\WebsocketImpl.rs

to proceed despite this and include the uncommitted changes, pass the `--allow-dirty` flag
after
Copy code
git add .
git commit -am "commit"
Copy code
warning: manifest has no description, license or license-file.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.gitignore
.vscode/settings.json
Cargo.toml
Cargo.toml.orig
public/index.html
rrutexws.code-workspace
src/StellarObjectImpl.rs
src/WebHandlerImpl.rs
src/WebsocketImpl.rs
src/lib.rs
src/test.rs
Do we need the git info on the server? Maybe we can add
Copy code
--allow-dirty
to the deploy process?
k
--allow-dirty
is already available as an arg to deploy: 
cargo shuttle deploy --allow-dirty
The cargo package lists look like they have the required files. I'll see if I can reproduce your error.
Hmm, I wasn't able to reproduce it. I created a new project, deleted .git and deployed. I then initialized git and committed all files. Deployment was still successful.
TLDR: I'm still not sure why this is happening with your project 😄
s
Hmm, @hundreds-market-43319 can you create a full archive using 
cargo package
. Then manually extract the file under 
target/package/*.crate
and see if the 
Cargo.toml
file is as the original?
h
Which format has .crate? Is it openable as zip/etc?
s
It's targz
h
omw 🙂
s
gzip -d
to deflate it
h
Seems like it same, yes. Formated by cargo but content is same
Deploy fails even for newly created project. I assuming this can be an pure Windows issue... But since package is creating and content is same... Hard to say.
s
What is the 
*.crate
file like after a 
cargo shuttle deploy
?
h
thats interesting... After shuttle deploy (previoused builded crate with cargo package) was overwrited by something with size of 1024 bytes and fills with 0.
crate inside first crate* fills with zeros.
s
Can you create a 
Secrets.toml
file in the repo with some dummy content and post the effects on 
*.crate
?
h
omw
Seems like it does affects anything: I created Secrets.toml, added some syntax-valid content and commited it into git. After i built successfully package from cargo and tried to deploy, content of intern crate was again filled with zeros.
s
What is the first few output lines of 
cargo shuttle deploy
? Ie the line starting with 
Archiving
h
cargo shuttle deploy whole log:
Copy code
warning: manifest has no description, license or license-file.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
warning: No (git) Cargo.toml found at `\\?\C:\dev_one\rrutexws\Cargo.toml` in workdir `C:/dev_one/rrutexws/`
   Packaging rutexws v0.1.0 (C:\dev_one\rrutexws)

2022-11-01T12:25:51.895274049Z  INFO Entering queued state

2022-11-01T12:25:51.896627462Z  INFO Entering building state
2022-11-01T12:25:51.952055012Z  INFO shuttle_deployer::deployment::queue: Extracting received data

Deployment crashed
Run the following for more details

cargo shuttle logs 17e30a21-384a-458e-9eb1-6a9aa2b634be
s
@kind-area-3810, what was that 
No (git) Cargo.toml found at '\\?\C:\
thing again?
h
idk... I have project on my c:\ and no any networks \ workspaces
maybe something wrong with path finding in shuttle binary itself?
k
I'm not sure if we have seen exactly that windows error before? I can't recall nor find it on discord/the github windows issue 🤔
Maybe relevant:
s
I found the "No (git)" in our August logs, but it did not impact deployment...
Does a normal 
cargo package
also jump from the "No (git)" warning to the "Packaging" line?
I'm expecting to see a bunch of "Archiving" lines between them...
h
I dont see any No (git) warnings... Only default warning from cargo like
Copy code
warning: manifest has no description, license or license-file.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
and after that line begins compiling
normal cargo package produces good .crate and no bugs/warning except default one. file corrupts only at the deploy...
s
What's your 
cargo version
?
h
Copy code
cargo 1.64.0 (387270bc7 2022-09-16)
s
Under the hood we just call the cargo library to get the 
.crate
archive. So I've hit a wall trying to figure out what is happening here...
Btw, does it work in WSL?
h
i dont have WSL installed at the moment, gonna check this out soon. But i guess it really depends on windows somehow...
Seems like hardcoding an
Copy code
working_directory
fixes issue...
the final crate is good too...
k
In that case, you could try setting the working directory with 
cargo shuttle deploy --working-directory
h
nope...
relative and full path in various combinations didnt gave anything
you can clearly see the difference between broken and good path.
Copy code
[cargo-shuttle\src\lib.rs:438] &working_directory = "\\\\?\\C:\\dev_one\\rrutexws"

[cargo-shuttle\src\lib.rs:442] &working_directory = "C:\\dev_one\\rrutexws"
I guess first path tries to access file path from an network path... if i understand everything correct. Or this is big strange bug
k
There are a few bugs with cargo and pathing on windows. I'm curious, what does the path look like after 
working_directory.join("Cargo.toml")
?
h
after hardcoding?
k
Yeah, and also without hardcoding but with the 
--working-directory
flag.
h
i didnt started server, only shuttle CLI. But after hardcoding (semi-removing
Copy code
\\\\?\\
) it generates correct crate, with valid content. I guess that was the issue. working_directory.join("Cargo.toml") produces valid path:
Copy code
[cargo-shuttle\src\lib.rs:441] &path = "C:\\dev_one\\rrutexws\\Cargo.toml"
i guess as a fix idea is to add filtering / cleaning to a RequestContext's working_directory function
i dont think we need full long styled paths... or at least for alpha / beta
Without hardcoding with an git repo.
k
For some dependencies I think we do need that.
Here is a tracking issue for all the issues with pathing on windows:
Also here is an alternative way to deploy on windows:
h
For example? Modern Languages path implementations can perfect handle any ways to work with paths. I see first time for many years that something depending on long path. And even if it does - mostly, it has an own implementation to deal with path.
I do not have docker and it uses massive amounts of resources on windows. I can't also work from wsl directly, since it's opt out and things like rls cannot work proper inside ssh bridge.
We can at least fix current issue with creating package by simple filtering path and leaving it untouched for other things. Shouldn't break anything global.
k
Don't get me wrong, it would be great to have this fixed, so if you have a good idea of how to do it a PR is very welcome (you have a lot more experience with windows pathing than I do 😅 )!
s
Out of interest, if you hardcode the path again and then send it through 
canonicalize
, is it broken again? https://doc.rust-lang.org/std/fs/fn.canonicalize.html
h
Yep, is it.
Copy code
canonicalize
transforms hardcoded traditional path into long styled path back and breaks everything again.
s
h
Well seems like this is working fix Gonna fix tests cases and create PR soon...
packaging git project from other directory good too.
s
Nice! Won't the windows version work on unix too?
h
Cfg target os compiles next expression only for target os. So If you compile on Linux first block is ignored. And in other direction it works too.
please run tests on your side 🙂 But it should work.
s
Oh, I meant replacing the 
canonicalize
with the code targeting windows? Ie removing the 
cfg
altogether
h
doesnt it will affect unix systems? like symlinks or full paths? Im not familiar with linux FS much
3 Views
PreviousNext
Powered by