Changing from &'static str to &'a str requires effort

[pickfire]

I was working with borrowed data such as &str when using nom. When I switch from &'static str (include_str!) to &'a str (&s which the data is fetched online), I expected no changes to very few changes, but I need to change a lot of things since Error in IResult does not keep the same lifetime.

Before (&'static str)

/// #3662,P[0.87|4017600]C[0.85|0.87|8940160|11962130|13091002],T11[3495312|18.93|87.097|55.357]#4995
pub fn parse_stock(input: &str) -> IResult<&str, Stock> {
    let (input, stock_id) = delimited(tag("#"), take_until(","), tag(","))(input)?;
    let (input, _) = tag("P[")(input)?;
    let (input, price) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, volume) = map_res(terminated(take_until("]"), tag("]")), str::parse)(input)?;
    let (input, _) = tag("C[")(input)?;
    let (input, open) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, close) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, _) = terminated(take_until("|"), tag("|"))(input)?;
    let (input, _) = terminated(take_until("|"), tag("|"))(input)?;
    let (input, _) = terminated(take_until("]"), tag("]"))(input)?;
    let (input, _) = tag(",T11[")(input)?;
    let (input, value) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, smi) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, power) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, power6m) = map_res(terminated(take_until("]"), tag("]")), str::parse)(input)?;

    let stock = Stock {
        stock_id,
        price,
        volume,
        open,
        close,
        value,
        smi,
        power,
        power6m,
    };
    Ok((input, stock))
}

/// 11265#3662,P[0.87|4017600]C[0.85|0.87|8940160|11962130|13091002],T11[3495312|18.93|87.097|55.357]#4995,...
pub fn parse_screenrule(input: &str) -> IResult<&str, ScreenRule> {
    let (input, rule_id) = map_res(take_until("#"), str::parse)(input)?;
    let (input, stocks) = terminated(many1(parse_stock), tag("\n"))(input)?;

    let screenrule = ScreenRule { rule_id, stocks };
    Ok((input, screenrule))
}

pub fn parse_screen(input: &str) -> IResult<&str, Screen> {
    // >
    // 2,20210304
    let (input, _) = tag(">\n")(input)?;
    let (input, _) = terminated(take_until(","), tag(","))(input)?;
    let parse_day = |s: &str| NaiveDate::parse_from_str(s, "%Y%m%d");
    let (input, day) = map_res(terminated(take_until("\n"), tag("\n")), parse_day)(input)?;

    let (input, rules) = many1(parse_screenrule)(input)?;

    let screen = Screen { day, rules };
    Ok((input, screen))
}

pub fn parse_screens(input: &str) -> IResult<&str, Vec<Screen>> {
    // Ignore H:tradingdays=2,11,20210305,20210304,20210303,20210302,20210301,20210226,20210225,
    let (input, _) = tag("H:tradingdays=")(input)?;
    let (input, _) = terminated(take_until(","), tag(","))(input)?;
    let (input, _) = terminated(take_until(","), tag(","))(input)?;
    let (input, _) = many_till(terminated(take_until(","), tag(",")), tag("\n"))(input)?;

    let (input, screens) = all_consuming(many1(parse_screen))(input)?;
    Ok((input, screens))
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let (_, screens) = parse_screens(&body)?;
    Ok(())
}

After (&str from fetched data), changes happens in how the root parser were called and the function signatures

/// #3662,P[0.87|4017600]C[0.85|0.87|8940160|11962130|13091002],T11[3495312|18.93|87.097|55.357]
fn parse_stock<'a, E>(input: &'a str) -> IResult<&'a str, Stock, E>
where
    E: ParseError<&'a str>
        + ContextError<&'a str>
        + FromExternalError<&'a str, ParseIntError>
        + FromExternalError<&'a str, ParseFloatError>,
{
    let (input, stock_id) = delimited(tag("#"), take_until(","), tag(","))(input)?;
    let (input, _) = tag("P[")(input)?;
    let (input, price) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, volume) = map_res(terminated(take_until("]"), tag("]")), str::parse)(input)?;
    let (input, _) = tag("C[")(input)?;
    let (input, open) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, close) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, _) = terminated(take_until("|"), tag("|"))(input)?;
    let (input, _) = terminated(take_until("|"), tag("|"))(input)?;
    let (input, _) = terminated(take_until("]"), tag("]"))(input)?;
    let (input, _) = tag(",T11[")(input)?;
    let (input, value) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, smi) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, power) = map_res(terminated(take_until("|"), tag("|")), str::parse)(input)?;
    let (input, power6m) = map_res(terminated(take_until("]"), tag("]")), str::parse)(input)?;

    let stock = Stock {
        stock_id,
        price,
        volume,
        open,
        close,
        value,
        smi,
        power,
        power6m,
    };
    Ok((input, stock))
}

/// 11265#3662,P[0.87|4017600]C[0.85|0.87|8940160|11962130|13091002],T11[3495312|18.93|87.097|55.357]...
fn parse_screenrule<'a, E>(input: &'a str) -> IResult<&'a str, ScreenRule, E>
where
    E: ParseError<&'a str>
        + ContextError<&'a str>
        + FromExternalError<&'a str, ParseIntError>
        + FromExternalError<&'a str, ParseFloatError>,
{
    let (input, rule_id) = map_res(take_until("#"), str::parse)(input)?;
    let (input, stocks) = terminated(many1(parse_stock), tag("\n"))(input)?;

    let screenrule = ScreenRule { rule_id, stocks };
    Ok((input, screenrule))
}

/// 11265#3662,P[0.87|4017600]C[0.85|0.87|8940160|11962130|13091002],T11[3495312|18.93|87.097|55.357]...
fn parse_screenrule<'a, E>(input: &'a str) -> IResult<&'a str, ScreenRule, E>
where
    E: ParseError<&'a str>
        + ContextError<&'a str>
        + FromExternalError<&'a str, ParseIntError>
        + FromExternalError<&'a str, ParseFloatError>,
{
    let (input, rule_id) = map_res(take_until("#"), str::parse)(input)?;
    let (input, stocks) = terminated(many1(parse_stock), tag("\n"))(input)?;

    let screenrule = ScreenRule { rule_id, stocks };
    Ok((input, screenrule))
}

fn parse_screen<'a, E>(input: &'a str) -> IResult<&'a str, Screen, E>
where
    E: ParseError<&'a str>
        + ContextError<&'a str>
        + FromExternalError<&'a str, ParseIntError>
        + FromExternalError<&'a str, ParseFloatError>
        + FromExternalError<&'a str, chrono::ParseError>,
{
    // >
    // 2,20210304
    let (input, _) = tag(">\n")(input)?;
    let (input, _) = terminated(take_until(","), tag(","))(input)?;
    let parse_day = |s: &str| NaiveDate::parse_from_str(s, "%Y%m%d");
    let (input, day) = map_res(terminated(take_until("\n"), tag("\n")), parse_day)(input)?;

    let (input, rules) = many1(parse_screenrule)(input)?;

    let screen = Screen { day, rules };
    Ok((input, screen))
}

fn parse_screens<'a, E>(input: &'a str) -> IResult<&'a str, Vec<Screen>, E>
where
    E: ParseError<&'a str>
        + ContextError<&'a str>
        + FromExternalError<&'a str, ParseIntError>
        + FromExternalError<&'a str, ParseFloatError>
        + FromExternalError<&'a str, chrono::ParseError>,
{
    // Ignore H:tradingdays=2,11,20210305,20210304,20210303,20210302,20210301,20210226,20210225,
    let (input, _) = tag("H:tradingdays=")(input)?;
    let (input, _) = terminated(take_until(","), tag(","))(input)?;
    let (input, _) = terminated(take_until(","), tag(","))(input)?;
    let (input, _) = many_till(terminated(take_until(","), tag(",")), tag("\n"))(input)?;

    let (input, screens) = all_consuming(many1(parse_screen))(input)?;
    Ok((input, screens))
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let (_, screens) = parse_screens::<()>(&body)?;
    Ok(())
}

structs here if you need it

#[derive(Debug)]
pub struct Screen<'a> {
    day: chrono::NaiveDate,
    rules: Vec<ScreenRule<'a>>,
}

#[derive(Debug)]
pub struct ScreenRule<'a> {
    rule_id: u32,
    stocks: Vec<Stock<'a>>,
}

#[derive(Debug)]
pub struct Stock<'a> {
    stock_id: &'a str,
    price: f64,
    volume: u64,
    open: f64,
    close: f64,
    value: u64,
    smi: f32,
    power: f32,
    power6m: f32,
}

Is there a way to not require typing out the &'a in every error? It took me roughly one hour to figure out why it suddenly didn't work when I changed from &'static str to &str, I happened to follow what nom did in the docs and I got it working. Not sure how we can improve this using rust type system.

Prerequisites

Here are a few things you should provide to help me understand the issue:

• Rust version : rustc -V rustc 1.52.0-nightly (e37a13cc3 2021-02-28)
• nom version : 6.1.2
• nom compilation features used:

[Geal]

I think the lifetimes could be elided if you used a real error type instead of making it generic over a E type.

[pickfire]

But that also requires me to put in all the types of the errors in there right? I really like the current behavior of error where I don't need to write it out, I don't even know how to create the type magic, I wish it will work well with lifetime as well, at least it makes it more consistent.

In some cases, we need the error type. In some cases, we don't need the error type. And that depends on the lifetime, would be confusing to new users. Maybe we should show an example of using the error type right away? Even then the ergonomics may not be that good.

[brownjohnf]

I think the lifetimes could be elided if you used a real error type instead of making it generic over a E type.

@Geal Could you elaborate on what you mean here? I've encountered this same issue. From what I understand, the issue is because the error types contain references to the input string. What would a "real error type" look like that would avoid this issue, ideally also without requiring copying the &str into something owned?

Sorry if this is a stupid question; I'm still learning Rust.

[brownjohnf]

To clarify, I've found and read https://github.com/Geal/nom/blob/master/doc/error_management.md, but this particular issue is still unclear to me. I'd be happy to PR a small example, if I can understand the solution.