Skip to content

Meeting 2013 06 07

Jump to bottom Edit New page
Gábor Lehel edited this page Jun 7, 2013 · 3 revisions

Agenda

  • closures
  • dynamic sized types
    • sized vs. unsized
  • GC

Attending

graydon, time, niko, bblum, patrick, brson, jclements

Dynamically Sized Types

   <T:Unsized> <-- anti-trait
   <unsized T> <-- 
   <?T>  <-- "Any T that you'd have to write ~T or @T on"
   (we will pick a syntax here)

    Gc<[T]> <-- @[T]
    Gc<Trait>
    Gc<str>

impl<T> Foo for &T { ... } <-- will only match

impl<unsized T> Foo for &T <-- if you want to match [T]

  • graydon: is a version that is marked up with Sized bounds somewhere we can look at it?
  • graydon: if this is an aesthetic call then we should see it
  • bblum: yes, pushing
  • graydon: don't feel like this is contentious. matter of figuring out the aesthetics
  • niko: yes

https://github.com/bblum/rust/tree/sized top three commits

Closures

  • niko: many facets to this issue. 1) so long as closure can mutate environment, for soundness we must prevent recursion, must be treated as linear

  • niko: i thought fn would be dst, but now don't agree. better fn is a value instead of reference to value, maybe a value that embeds a borrowed pointer to environment, but better to treat it like a value

  • niko: adv is that you can move it arround - borrowed pointers are always reborrowable, a property you don't want. if you want a linear pointer you need &mut fn(), which feels wrong here

  • graydon: too much chatter in fn sig

  • niko: there are many solutions. 2 concrete proposals, mine (closures lite), and bblum's (closures max)

  • bblum's proposal: https://github.com/mozilla/rust/wiki/Proposal-for-closure-reform-%28specific%29 (-ed)

  • niko: in mine you have type 'fn', always stack allocated, etc:

  • fn(T*) -> T ==> always a stack-allocated closure always references local variables of some enclosing stack

  • fn:'r K(T*) -> T ^^ region bound ^^ environment bound

  • niko: very little code today uses bounds on fns. use cases include scheduler and data parallel stuff. places where you want other tasks to refer to remote stack allocations

  • niko: have to address a few other use cases. proposed the 'thunk' macro, expands to a struct that implements a trait

  • niko: did reserach into that. found that in std/extra ,thunk covers vast majority of ~fn. a few places that wouldn't be covered. most were environmentless fns in libuv that already take user-data. others I thought could be encoded as traits

  • niko: i feel like this covers a lot of use cases. will be a lot of places that you would need a trait where you would have written an @fn before

  • bblum: the summary of my proposal is to try to keep heap closures in the language. probably involve writing with pointer sigil after fn to indicate that while traits, vecs, are pointers, fns are special - values with the pointer inside

  • bblum: proposal optionally unifies stack/heap closures wrt wether they own their env.

  • bblum: eliminate the way stack closures implicitly ref env. when stack closure references higher stack frame it's actually creating an explicit reference and owning it. If you were going to access a borrowed upvar, it would have type &T where T is the declared type of the variable.

  • bblum: to do this in a sane way we need to introduce capture clauses, have defaults. the default for stack closure is create borrowed pointers, default for heap is copy env. wouldn't stop you from writing heap closure that captures ref x, ref y and doesn't satisy kind bounds.

  • bblum: heap closures can't be promoted to stack closures, except by writing |x| f(x)

  • fn&:'r K(...) fn@:'r K(...) fn~:'r K(...)

  • niko: I don't think it unifies whether closure own environment, but it does make all closures copying, the thing being copied may be borrowed pointers.

  • bblum: another thing, we can change heap closure literals to require a tilde behind them, do spawn ~{ .. }. prevents need for closure inference. shows allocation. (or an @, if we keep those)

  • niko: I think both proposals make sense, however I'm inclined to try to simplify things so we prevent smallest surface area w/o sacrificing critical expressiveness.

  • niko: sigils after fns is confusing. there would be other syntaxes. main difference is whether to keep heap closures or make borrowed pointers explicit

  • graydon: can we get everyone's reactions?

  • jclements: i feel like felix has delegated to me to convey his concern about explicit capture clauses, the idea that a macro could introduce freevars that the user might not want to have to know about. any solution that requires capture clauses could cause issues for macros.

  • niko: you are saying that in some fn:

fn foo() { some_macro!(...); // introduced a gensym'd name ... thunk!( /* would have to name this item above */ some_other_macro(); // references that name

  • niko: this presumably won't be global state since we don't have it
  • jclements: yeah it's possible that that might make this ok, felix had in mind structure names, but those are ok because they are not variable references. we don't have generative structures - i.e. if I evaluate a struct decl multiple times are they distinct?
  • niko: we don't evaluate structures
  • jclements: in the absence of globals i agree
  • brson: we have globlas
  • niko: we do have unsafe ones, but the only things that need to be captured anyway are in function scope
  • jclements: yeah, let's go back to your example .... hard to imagine that coming up
  • niko: I feel like I wouldn't object to the requirement that the macro returns the value and the user gives it a name
  • graydon: this is going to have the same problem if you do anything involving code that makes use of a name generated by macro
  • graydon: like if you pass it to a constructor function you have the same function
  • jclements: don't understand, but I'd say 'no', hygien lets you deal with those kinds of things
  • jclements: let's abandon this and just say I can imagine explicit capture clauses down the road causing problems with macros, but it may not be a significant enough problem to prevent thissolution
  • graydon: k
  • brson: I want to retain the ability to write wrappers, here are some examples:

https://github.com/brson/rust/blob/io/src/libstd/rt/test.rs#L42

  • pcwalton: I'm pretty strongly in favor of Niko's proposal, I am concerned about the proliferation of sigils, I find multiple fn/pointer/etc types to be largest stumbling block. It seems much more approachable to have only one fn type. I am concerned about making sure that whatever system we come up with accommodates the use cases, since first and foremost we are creating a language for us to use, but at the end of the day I am concerned about confusion due to the proliferation of sigils/memory-management annotations and it just seems to me that having fn as solely used for callback patterns...the one thing that would concern me is if we wound up being like Java where everything has to be wrapped in a class type. Closures do present some memory management difficulty that we either surface with syntax or don't, and I'd prefer to err on the side of simplicity.

  • graydon: Keep in mind that you have to do pretty astonishinly verbose things in Java, so I think we're not talking about that level of verbosity.

  • pcwalton: Yeah ihttps://github.com/mozilla/rust/wiki/Proposal-for-closure-reform-%28specific%29t's just stuff like new ActionListener { public void actionPerformed() { ... } }

  • graydon: it just seems to me that a keyword and a list of the variables that are captured is much lighter

  • graydon: I'm assuming something like this, though I don't think the word thunk is going to be the most popular choice, but let's go with it for a second:

  • thunk!((a, b, c), |p, q| a + b + c);

  • GC::new(thunk!((a, b, c), |p, q| a + b + c));

  • RC::new(thunk!((a, b, c), |p, q| a + b + c));

  • pcwalton: with a macro we can certainly fiddle with the capture/argument syntax a lot

  • graydon: I just don't think you're talking about large, multi-line decls like Java

  • pcwalton: Given Rust's explicit approach to memory management, you will probably want to have all kinds of callbacks, some ref-counted, some whatever, seems hard to fit that into a proposal with a fixed set of sigils.

  • pcwalton: Trait doesn't have these issues since you already allocated the space, all you do is box the value up with a vtable.

  • bblum: How does that make the allocation story easier?

  • pcwalton: Basically you're just allocated the struct in the normal way, it's not existential at this point, and then you convert it to an opaque trait type. Just basically pairs up the allocation with a vtable, so it is easier in that sense to have custom smart pointers and things like that.

  • bblum: I don't understand why this makes allocation story easier than any other proposal

  • graydon: Maybe we're not talking about the same thing, what I'm hearing pcwalton discussing is that in Niko's proposal, which doesn't have any form of owning capture, there is nothing to think about.

  • bblum: You think about in emulation using this macro.

  • graydon: Yeah but the compiler and type system dont' do any additional thinking, the emulation is just moving into the value as you would with any value

  • bblum: It's a big important use case that we're deciding to use a macro for, instead of making the type system rise to the challenge

  • niko: disagree, object types are more expressive. macros are bridging the gap between closures and objects. that said, there aren't theoretical limitations, but there are things that will be harder with my proposal. like to enumerate them. I don't hate java syntax or at least am more tolerant than others.

  • niko: I originally proposed trait 'Thunk'. it makes assumptions - takes no arguments, run once. if you want arguments you add type params, like Thunk1<A, R>. Could use a tuple to pass multiple arguments or add Thunk2.

trait Thunk<R> {
    fn run(self) -> R;
}

trait Thunk1<A,R> {
    fn run(self, a: A) -> R;
}
  • niko: This is still less expressive. If I write something like:

fn<'a>(x: &'a Foo) <-- with a fn type, you can bind a lifetime lifetime is instantiated when you call it

  • niko: Can bind lifetime in the function type, bound when called.
  • niko: Can't have an equivalent thunk.

forall 'b. Thunk1<&'b Foo> <-- bound up for caller Thunk1<&'c Foo> <-- 'c must be some lifetime already in scope

  • niko: This issue comes up with conditions. if we had a standard thunk trait it would probably look like Thunk1, but it's limited. Can't pass all args that you pass to ~fn. Can't pass references. If you had that need you would need a different trait or environment-less function that takes an environment argument
  • niko: reason for favoring macros - other libraries may want their own variations on 'Thunk', 'ActionListenerCallbackThunk'. Thunk wouldn't apply because ActionListerCallback is invoked many times.
  • niko: Could imagine having one or two in std, and people would need to evolve their own. I find that acceptable, but understand why otherswouldnet'
  • niko: That's the main limitation, all other examples boil down to this lifetime problem.
  • graydon: Everyone understand?

  • graydon: Do you feel these are problems that can be worked around in a tolerable way?

  • brson: I'm willing to try

  • graydon: I kind of feel the same as patrick. Solving this problem in a way that does the least possible and removes the most 'things' is really strongly preferrable right now. Can't see all the implications of how this fits together and we're people that have been looking at this for years. Putting all these options in the language or standard library will be just as opaque as any mess of macros would be.

  • graydon: tend toward the version that says is just an upvar capture

  • graydon: lots of languages survive with no lambda construct at all. big codebases. their awkward, but where the awkwardness comes up is in throwaway functions, small stack closures with captures.

  • niko: if you're using traits you get to push the choice of static/dynamic dispatch to the caller.

  • jclements: thunk! would be generating fresh trait?

  • niko: no

  • niko: would be generating an instance of a struct type

  • graydon: particularly for things like spawn, we want spawn to identify where it comes from in source, so it's going to be macroized anyway.

  • niko: it's not minor to talk about brian's use cases where we crate a million wrappers around spawn

  • pcwalton: it's going to happen all the time

  • graydon: but the think that they're spawning, we can standardize a type for that, 'a thing that is ready to go as a process'

  • niko: we can prototype this today with no changes. the only problem is there might be bugs in ~objects

  • niko: otherwise it's not adding anything, just removing

  • graydon: I suggest going down that road. everyone ok with that?

  • niko: I actually think that DST may help in the condition case. we'll revisit.

  • niko: It's not conditions that are unsound it's closures
  • niko: the borrow checker needs to model

unsound conditions: https://github.com/mozilla/rust/issues/6459

  • niko: last question here is once fns.

GC

  • graydon: there's a thread on the list about GC. someone is saying we should try to statically prevent rc cycles
  • pcwalton: we tried that
  • graydon: what terrifies me about that is that a lot of live systems that are really robust are refcounted and a lot of live GC's are not the smart ones, they're pausy and use lots of memory
  • pcwalton: does anyone actually use Azul C4?
  • graydon: no
  • pcwalton: it's like the legend of Chez Scheme's incredible speed. it costs too much money, nobody uses
  • graydon: pcwalton wants to libify everything. annoyed because I wish you would outline how, not the why
  • graydon: sigil argument isn't persuasive. when languages don't specify memory management scheme it's sad times.
  • pcwalton: hasn't been tried much
  • graydon: thinking ada specifically. they are hammered because they don't specify RC vs GC, so no ada libraries know the semantics, etc.
  • pcwalton: also C++ though in practice no GC impl exists or likely will
  • pcwalton: I do think that actually having the programmer notate the semantics they want may alleviate that problem, you have Gc, Rc
  • pcwalton: agree i should have specified how this works
  • graydon: I don't know how you can do this. if you can librarify I'm in favor. seems non-contentious to move code out of compiler
  • pcwalton: take lang items for GC into a trait. borrowck and trans emit calls to trait functions. it isn't likely this will allow two tracing GC's in one program.
  • graydon: they need to know about each other
  • pcwalton: and the memory management system needs to know about GC to some extent
  • pcwalton: i'm interested in a more narrow set of use cases, mixing GC/RC/ARC.
  • pcwalton: want to be able to mix them and omit GC entirely while not feeling like you are working against the language
  • pcwalton:
  • pcwalton: llvm's gc is a general facility for recording stack addresses of 'things'
  • niko: is it accurate to summarize: essentially we are talking about not supporting arbitrary number of GC's, GC is a privileged library, it has some special hooks to make it work. problem of ~T that owns @T has special hacks would be extended to work with Rc, etc.?
  • pcwalton: yes. it's not a platform for research into GC. one of the issues on the mailing list is that we should provide it in the standard library, make it clear that this is the GC to use
  • niko: seems like the only other options are a conservative scan. hooks won't be sufficiently general
  • pcwalton: in practice, when people write libs in rust i would use ~ where I can and when needing dynamic lifetimes use Rc. It's usually a small subset of data that needs it and it's the least objectionable. Interop with external RC systems is common.
  • niko: doubly linked list is not a good place for RC since it's cyclic
  • pcwalton: yes, but you won't have problems with ... yeah it's problematic. in any case it's solvable
  • graydon: we already have an Rc type?
  • pcwalton: yes
  • graydon: what are the current inadequancies?
  • pcwalton: doesn't integrate with borrowck like GC. needs closures for borrowing
  • graydon: totally fixable?
  • pcwalton: yes, by making the language more GC agnostic
  • graydon: more of GC functionality defers to libraries. don't want to keep everyone here for hours discussing something we agree on.
  • graydon: what's the contentious bit? @
  • pcwalton: I think so
  • graydon: anyone with strong feelings?
  • niko: at first I thought it was crazytown but now I like the word 'Gc`.
  • graydon: you are moderately in favor of removing @
  • jclements: there's a big diff between two-letter Gc and something big and ugly. agree Gc isn't onerous.
  • graydon: have to write something like:

Gc::new(foo) vs. @foo

  • niko: needs a 'new' keyword:

    new(Gc) [1, 2, 3, 4, 5]

  • niko: DST's require it

  • pcwalton: Gc::new would require a move

  • niko: we do it today with closures

    new(GC) Foo::New(1, 2, 3) @Foo::New(1, 2, 3) Rc::alloc(|| Foo::new(1, 2, 3))

  • bblum: think this is argument in favor of @

  • pcwalton: disagree

  • graydon: this is straight-outta-C++

<i'm not following the convo>

  • alloc GC Foo::new(1,2,3)
  • alloc GC [1, 2, 3] <-- DST
  • alloc [1, 2, 3] <-- ?
flow_arena.alloc(|| InlineFlow(whatever))
flow_arena.alloc_move((a, b), |(a, b)| InlineFlow(whatever, a, b))
  • niko: think we're done here. will iterate on the syntax for separating allocation and construction, try to prototype Thunk, continue checking into ways of making Sized annotation not too onerous, measuring, if necessary return to question of an 'unsized' keyword or annotation.
Add a custom footer

All Categories:

Clone this wiki locally