Skip to content

Conversation

jakecorrenti
Copy link
Member

@jakecorrenti jakecorrenti commented Jun 2, 2025

Vfkit supports the schemes unix and none in addition to tcp.
Krunkit currently only implements tcp, so implement the remaining
schemes to meet vfkit parity.

Resolves #47

@jakecorrenti jakecorrenti marked this pull request as ready for review June 4, 2025 21:40
@jakecorrenti jakecorrenti requested a review from slp as a code owner June 4, 2025 21:40
@jakecorrenti jakecorrenti changed the title [WIP] Add missing URI schemes Add missing URI schemes Jun 4, 2025
@jakecorrenti
Copy link
Member Author

@nirs let me know if this works for the use case you were asking for. Thanks!

Copy link

@nirs nirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'll test it.

Copy link

@nirs nirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand the rust details. The unix support looks good, but parsing tcp looks wrong now.

I will test it tomorrow.

@jakecorrenti jakecorrenti force-pushed the restful-uri-parity branch 2 times, most recently from 5f23366 to 1f290b8 Compare June 5, 2025 20:30
@nirs
Copy link

nirs commented Jun 7, 2025

https://github.com/containers/krunkit/blob/main/docs/usage.md should be updated to describe the new schemes.

nirs added a commit to nirs/vmnet-helper that referenced this pull request Jun 7, 2025
Depends on containers/krunkit#51 adding `unix`
and `none` schemes. Since we don't use the listener yet we can disable
it.
nirs added a commit to nirs/minikube that referenced this pull request Jun 7, 2025
This removes the limit of using a single machine, removing the need to
manage tcp ports. We use:

    --restful-uri unix://$MINIKUBE_HOKE/machines/NAME/krunkit.sock

The socket is created and deleted by krunkit.

NOTES:
- Depends on containers/krunkit#51.
  To test this change build and install krunkit from this PR.
@nirs
Copy link

nirs commented Jun 7, 2025

Testing with nirs/vmnet-helper#86, krunkit panics.

From krunkit.log:

thread 'main' panicked at src/status.rs:82:22:
internal error: entered unreachable code
stack backtrace:
   0:        0x102b5abbc - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h39ba3129e355bb22
   1:        0x102b71060 - core::fmt::write::h8b50d3a0f616451a
   2:        0x102b58430 - std::io::Write::write_fmt::h4b3bbae7048e35f8
   3:        0x102b5aa70 - std::sys::backtrace::BacktraceLock::print::h7934b1e389160086
   4:        0x102b5b9bc - std::panicking::default_hook::{{closure}}::hbcd636b20f603d1e
   5:        0x102b5b7f0 - std::panicking::default_hook::ha9081970ba26bc6c
   6:        0x102b5c1ac - std::panicking::rust_panic_with_hook::h9a5dc30b684e2ff4
   7:        0x102b5bdfc - std::panicking::begin_panic_handler::{{closure}}::hbcb5de8b840ae91c
   8:        0x102b5b080 - std::sys::backtrace::__rust_end_short_backtrace::ha657d4b4d65dc993
   9:        0x102b5badc - _rust_begin_unwind
  10:        0x102b7f60c - core::panicking::panic_fmt::hda207213c7ca0065
  11:        0x102b7f67c - core::panicking::panic::h58f3db1cc4835fac
  12:        0x102a06778 - <krunkit::status::RestfulUri as core::str::traits::FromStr>::from_str::h0af2136fbff640c9
  13:        0x102a12458 - <F as clap_builder::builder::value_parser::TypedValueParser>::parse_ref::hbb6591a7d4969ec1
  14:        0x102a10b74 - <P as clap_builder::builder::value_parser::AnyValueParser>::parse_ref_::hce1866269762cfb7
  15:        0x102b06738 - clap_builder::parser::parser::Parser::push_arg_values::hf902b32575cdb4a8
  16:        0x102b072b8 - clap_builder::parser::parser::Parser::react::h17b205031d96c887
  17:        0x102b06394 - clap_builder::parser::parser::Parser::parse_opt_value::h7136b4f4159d8ab9
  18:        0x102b015d0 - clap_builder::parser::parser::Parser::get_matches_with::h38f1a3f6a0ac1efb
  19:        0x102b0a3f4 - clap_builder::builder::command::Command::_do_parse::h192bfa12595d75c6
  20:        0x102a19d38 - clap_builder::builder::command::Command::get_matches_from::hd7afad657ceed3ca
  21:        0x102a126d0 - clap_builder::derive::Parser::parse::h98ed4cea26b1284a
  22:        0x102a20bf4 - krunkit::main::h3d66209327d81822
  23:        0x102a20d4c - std::sys::backtrace::__rust_begin_short_backtrace::h10b1bd8337916ca9
  24:        0x102a20ccc - std::rt::lang_start::{{closure}}::h6e97a35bf24caab2
  25:        0x102b53380 - std::rt::lang_start_internal::hf1bc3d9041088441
  26:        0x102a20ca8 - _main

The krunkit command:

krunkit \
    --memory=1024 \
    --cpus=1 \
    --restful-uri=none:// \
    --device=virtio-blk,path=/Users/nir/.vmnet-helper/vms/server/disk.img \
    --device=virtio-blk,path=/Users/nir/.vmnet-helper/vms/server/cidata.iso \
    --device=virtio-net,unixSocketPath=/Users/nir/.vmnet-helper/vms/server/vmnet.sock,mac=92:c9:52:b7:6c:08 \
    --device=virtio-serial,logFilePath=/Users/nir/.vmnet-helper/vms/server/serial.log \
    --krun-log-level=3

This diff fixes the panic:

--- a/src/status.rs
+++ b/src/status.rs
@@ -79,14 +79,10 @@ impl FromStr for RestfulUri {
                     }
                     Ok(Self::Unix(postfix))
                 }
-                _ => unreachable!(),
+                UriScheme::None => Ok(Self::None),
             }
         } else {
-            if UriScheme::None != UriScheme::from_str(s)? {
-                return Err(anyhow!("invalid scheme input"));
-            }
-
-            Ok(Self::None)
+            Err(anyhow!("invalid resful-uri"))
         }
     }
 }

With the fix I tested starting 2 vms with --restful-uri=none:// and running iperf3 between the vms.

I also tested with current minikube krunkit driver using tcp://localhost:8981
kubernetes/minikube#20826

Testing show that sending Stop command does nothing, and krukit is not terminated.

I can reproduce using curl:

% curl 'http://localhost:8081/vm/state' -o -        
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

% curl -X POST 'http://localhost:8081/vm/state' -o -       
{"state": "VirtualMachineStateStopping"}%                                                                                                                        

% curl -X POST 'http://localhost:8081/vm/state' -o -
{"state": "VirtualMachineStateStopping"}%                                                                                                                        

% curl 'http://localhost:8081/vm/state' -o -        
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

In krubnkit log we don't see any error from the the listener. I guess this is an issue in libkrun, not handling the event fd.

Finally I change minikube krunkit driver to use unix:// uri:
kubernetes/minikube#20900

I see the same behavior, krunkit does not terminate.

I could also reproduce the issue with curl:

% curl --unix-socket ~/.minikube/machines/minikube/krunkit.sock http://_/vm/state -o-
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

% curl -X POST --unix-socket ~/.minikube/machines/minikube/krunkit.sock http://_/vm/state -o-
{"state": "VirtualMachineStateStopping"}%                                                                                                                        

% curl --unix-socket ~/.minikube/machines/minikube/krunkit.sock http://_/vm/state -o-        
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

% curl -X POST --unix-socket ~/.minikube/machines/minikube/krunkit.sock http://_/vm/state -o-
{"state": "VirtualMachineStateStopping"}%

In summary, this PR is almost right, but shutting down the guest does not work and users need to use signals. I guess podman does not use --restul-uri?

@nirs
Copy link

nirs commented Jun 7, 2025

I did another test starting krukit from this command:

krunkit \
    --memory 6000 \
    --cpus 2 \
    --restful-uri tcp://localhost:8081 \
    --device virtio-blk,path=/Users/nir/.lima/uvkc/diffdisk \
    --device virtio-blk,path=/Users/nir/.lima/uvkc/cidata.iso \
    --device virtio-net,unixSocketPath=/Users/nir/.lima/uvkc/vmnet-helper.sock,mac=92:c9:52:b7:6c:08

This vm is using Fedora 42. Previous test use a minikube vm.

This time sending a POST request works and krunkit was terminated.

curl commands:

% curl http://localhost:8081/vm/state -o-
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

% curl -X POST http://localhost:8081/vm/state -o-
{"state": "VirtualMachineStateStopping"}%                                                                                                                        

% curl -X POST http://localhost:8081/vm/state -o-
curl: (7) Failed to connect to localhost port 8081 after 0 ms: Couldn't connect to server

krunkit stderr:

[2025-06-07T22:36:51Z INFO  vmm::macos::vstate] vCPU 0 received shutdown signal
[2025-06-07T22:36:51Z INFO  vmm] Vmm is stopping.

So it seems that krunkit is doing the right thing, but maybe the minikube vm is missing something that ignores the termination request.

The interesting thing is that the same vm does terminate with vfkit. Maybe Apple virtualization framework uses another way to terminate the VM.

@jakecorrenti
Copy link
Member Author

<snip>

With the fix I tested starting 2 vms with --restful-uri=none:// and running iperf3 between the vms.

I also tested with current minikube krunkit driver using tcp://localhost:8981 kubernetes/minikube#20826

Testing show that sending Stop command does nothing, and krukit is not terminated.

I can reproduce using curl:

% curl 'http://localhost:8081/vm/state' -o -        
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

% curl -X POST 'http://localhost:8081/vm/state' -o -       
{"state": "VirtualMachineStateStopping"}%                                                                                                                        

% curl -X POST 'http://localhost:8081/vm/state' -o -
{"state": "VirtualMachineStateStopping"}%                                                                                                                        

% curl 'http://localhost:8081/vm/state' -o -        
{"state": "VirtualMachineStateRunning"}%                                                                                                                         

In krubnkit log we don't see any error from the the listener. I guess this is an issue in libkrun, not handling the event fd.

Finally I change minikube krunkit driver to use unix:// uri: kubernetes/minikube#20900

<snip>

In summary, this PR is almost right, but shutting down the guest does not work and users need to use signals. I guess podman does not use --restul-uri?

Hi @nirs, thank you very much for the thorough testing. I tried this with podman by doing the following:

❯ bin/darwin/podman machine init --now --log-level=debug
# get the port for the restful service from debug console
❯ curl 'http://localhost:49748/vm/state' -o -
{"state": "VirtualMachineStateRunning"}%

❯ curl -X POST 'http://localhost:49748/vm/state' -o -
{"state": "VirtualMachineStateStopping"}%

❯ curl 'http://localhost:49748/vm/state' -o -
curl: (7) Failed to connect to localhost port 49748 after 0 ms: Couldn't connect to server

❯ bin/darwin/podman machine inspect --format '{{ .State }}'
stopped

So it seems to work with Podman at least

@nirs
Copy link

nirs commented Jun 9, 2025

@jakecorrenti I tested it with a modified minikube kernel using default configuration for the architecture and it works.

Copy link

@nirs nirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

Copy link

@nirs nirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks right but the tests are broken.

This diff fixes the tests:

--- a/src/status.rs
+++ b/src/status.rs
@@ -70,7 +70,12 @@ impl FromStr for RestfulUri {
                 let (ip_addr, port) = parse_tcp_input(value)?;
                 Ok(Self::Tcp(ip_addr, port))
             }
-            UriScheme::Unix => Ok(Self::Unix(value.to_string())),
+            UriScheme::Unix => {
+                if value == "" {
+                   return Err(anyhow!("empty unix socket path"));
+                }
+                Ok(Self::Unix(value.to_string()))
+            }
             UriScheme::None => Ok(Self::None),
         }
     }
@@ -201,7 +206,7 @@ mod tests {
         assert_eq!(true, RestfulUri::from_str("tcp:///tmp/path").is_err(),);
 
         // none valid
-        assert_eq!(RestfulUri::None, RestfulUri::from_str("none").unwrap(),);
+        assert_eq!(RestfulUri::None, RestfulUri::from_str("none://").unwrap(),);
 
         // random string
         assert_eq!(true, RestfulUri::from_str("foobar").is_err(),);

// unix valid
assert_eq!(
RestfulUri::Unix("/tmp/path".to_string()),
RestfulUri::from_str("unix:///tmp/path").unwrap()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this errors, the test panics and we don't get any info on the failure.

I tested this change, simulating a bug in the code:

diff --git a/src/status.rs b/src/status.rs
index c0e889e..b8520b0 100644
--- a/src/status.rs
+++ b/src/status.rs
@@ -179,7 +179,7 @@ mod tests {
         // unix valid
         assert_eq!(
             RestfulUri::Unix("/tmp/path".to_string()),
-            RestfulUri::from_str("unix:///tmp/path").unwrap()
+            RestfulUri::from_str("uni:///tmp/path").unwrap()
         );
 
         // unix missing path
% cargo test
...
test status::tests::uri_input_parsing ... FAILED

failures:

---- status::tests::uri_input_parsing stdout ----
thread 'status::tests::uri_input_parsing' panicked at src/status.rs:182:53:
called `Result::unwrap()` on an `Err` value: invalid scheme input
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    status::tests::uri_input_parsing

test result: FAILED. 9 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

error: test failed, to rerun pass `--bin krunkit`

The test cannot assume that the code does not return an error, it must validate this.

Copy link
Member Author

@jakecorrenti jakecorrenti Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I fully understand. Are you suggesting not using .unwrap in the test when trying to assert a successful parse?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if we fail to parse we want to report an error. If we parsed and got the wrong results we want to report the wrong result. If in both case we stop running the current test this will not make much difference.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thinks is fine now since we have single assert per test, and the assert reports the failure with the error message or the mismatched value.

// tcp valid
assert_eq!(
RestfulUri::Tcp(Ipv4Addr::new(127, 0, 0, 1), 8080),
RestfulUri::from_str("tcp://localhost:8080").unwrap(),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same - error will break the test instead of failing a single test.

@nirs
Copy link

nirs commented Jun 10, 2025

Looks like we are missing basic CI running the tests for every PR. We can assume that developers run the tests but it is much easier when CI runs the tests for you.

Copy link

@nirs nirs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@jakecorrenti
Copy link
Member Author

@nirs I updated the tests so that if a Result type returns either an unexpected Ok or Err, it will give a more detailed message and not just panic.

Let me know what you think!

src/status.rs Outdated

#[test]
fn parse_valid_unix_scheme() -> Result<(), String> {
let scheme = "unix:///tmp/path";
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a scheme, use "input"?

src/status.rs Outdated
let uri = RestfulUri::from_str(scheme);
assert_ok!(scheme, uri)?;
assert_eq!(RestfulUri::Unix("/tmp/path".to_string()), uri.unwrap(),);
Ok(())
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to return a Result? This makes the tests more verbose and complicated.

I think the previous tests were better. They were clear, short, and direct. Now we have temporary variables, hard to follow macros, and unclear Result.

Lets look at the previous test output and see if something is missing:

Simulating wrong path:

--- a/src/status.rs
+++ b/src/status.rs
@@ -182,7 +182,7 @@ mod tests {
     #[test]
     fn parse_valid_unix_scheme() {
         assert_eq!(
-            RestfulUri::Unix("/tmp/path".to_string()),
+            RestfulUri::Unix("/tmp/pathx".to_string()),
             RestfulUri::from_str("unix:///tmp/path").unwrap()
         );
     }

Result:

---- status::tests::parse_valid_unix_scheme stdout ----
thread 'status::tests::parse_valid_unix_scheme' panicked at src/status.rs:184:9:
assertion `left == right` failed
  left: Unix("/tmp/pathx")
 right: Unix("/tmp/path")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Looks good, no change needed.

Simulating failed parse:

diff --git a/src/status.rs b/src/status.rs
index 0f925e7..f50ce58 100644
--- a/src/status.rs
+++ b/src/status.rs
@@ -183,7 +183,7 @@ mod tests {
     fn parse_valid_unix_scheme() {
         assert_eq!(
             RestfulUri::Unix("/tmp/path".to_string()),
-            RestfulUri::from_str("unix:///tmp/path").unwrap()
+            RestfulUri::from_str("uni:///tmp/path").unwrap()
         );
     } 

Result:

---- status::tests::parse_valid_unix_scheme stdout ----
thread 'status::tests::parse_valid_unix_scheme' panicked at src/status.rs:186:53:
called `Result::unwrap()` on an `Err` value: invalid scheme input
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

We see the unexpected error message, good enough.

src/status.rs Outdated
anyhow!(expected_err).to_string(),
uri.err().unwrap().to_string()
);
Ok(())
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simulating unexpected successful parse with previous version:

diff --git a/src/status.rs b/src/status.rs
index 0f925e7..cc0bb3d 100644
--- a/src/status.rs
+++ b/src/status.rs
@@ -191,7 +191,7 @@ mod tests {
     fn parse_unix_scheme_missing_path() {
         assert_eq!(
             anyhow!("empty unix socket path").to_string(),
-            RestfulUri::from_str("unix://").err().unwrap().to_string()
+            RestfulUri::from_str("unix://foo").err().unwrap().to_string()
         );
     }
 

Result:

---- status::tests::parse_unix_scheme_missing_path stdout ----
thread 'status::tests::parse_unix_scheme_missing_path' panicked at src/status.rs:194:54:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Not very helpful, the error message is about wrong usage instead of describing that parsing did not fail.

We can simplify the negative tests like this:

diff --git a/src/status.rs b/src/status.rs
index 0f925e7..59cabd3 100644
--- a/src/status.rs
+++ b/src/status.rs
@@ -188,11 +188,9 @@ mod tests {
     }
 
     #[test]
+    #[should_panic(expected = "empty unix socket path")]
     fn parse_unix_scheme_missing_path() {
-        assert_eq!(
-            anyhow!("empty unix socket path").to_string(),
-            RestfulUri::from_str("unix://").err().unwrap().to_string()
-        );
+        RestfulUri::from_str("unix://foo").unwrap();
     }
 

The code is much simpler and we get a reasonable error:

---- status::tests::parse_unix_scheme_missing_path stdout ----
note: test did not panic as expected

What do you think?

Vfkit supports the schemes `unix` and `none` in addition to `tcp`.
Krunkit currently only implements `tcp`, so implement the remaining
schemes to meet vfkit parity.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
@jakecorrenti
Copy link
Member Author

@nirs since you approved the previous revision I reverted back to that. I'll come back in with another PR to address your comments on the tests. I think we should just move this PR through if it was in an ok state prior.

Copy link
Member

@tylerfanelli tylerfanelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Will wait for @nirs approval.

@nirs
Copy link

nirs commented Jun 10, 2025

@nirs since you approved the previous revision I reverted back to that. I'll come back in with another PR to address your comments on the tests. I think we should just move this PR through if it was in an ok state prior.

Agree, time to merge.

@tylerfanelli tylerfanelli merged commit 3a369ee into containers:main Jun 10, 2025
3 checks passed
nirs added a commit to nirs/vmnet-helper that referenced this pull request Jul 6, 2025
Depends on containers/krunkit#51 adding `unix`
and `none` schemes. Since we don't use the listener yet we can disable
it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support --restful-uri unix:/path like vfkit
4 participants