Use Public/Restricted Beta Testing for FuniNOW



  • …I reposted this debate I started in another thread, as it merits its own discussion nonetheless:

    @thegrandalliance:

    …preeeety sure this situation (Funimaiton not using Public Betas) is more of the lack of the software developer/programmer's experiential competence to run a beta, then the subscriber's ability to make use thereof...

    IN ENGLISH: That means Funimation doesn't know WTF how to run a beta; nor what is a public beta is for that matter...

    Evidence: Release of FUNimation NOW! APP... and all of the "bugs it did not have" of course...

    @Sophie:

    It's a little bit of both. We have run a beta before. In fact, we ran it for this site and for the old Drupal site.

    In this case, user testing as part of a public beta isn't really beneficial to us because users usually don't know the difference between actual bugs and content issues without knowledge of the requirements. It's a lot faster for a core QA team to do it before launch.

    Even with the new apps, users have only reported like 2 or 3 things that we didn't already know about that weren't content issues. The same thing has happened on the UK and IE sites.

    At best, users can tell us about features that they want, what of the app that they do or don't like, and anything they are confused by, which is what we are getting now. They can also help us to gather data when we aren't able to reproduce an issue. All of this is extremely valuable, but isn't necessarily anything to delay a launch over, which is what would have happened.

    We've never said that the FunimationNow app doesn't have bugs and we would not want to make such a claim, especially when it is not true. Please do let us know if you see anything like that, so that we can look into it.

    As to the response:

    @Sophie:

    In this case, user testing as part of a public beta isn't really beneficial to us because users usually don't know the difference between actual bugs and content issues without knowledge of the requirements. It's a lot faster for a core QA team to do it before launch.

    ….......

    Although it is true some people misinterpret a "beta" as "final release", and they whine and complain anyways... the solution to this is to have a closed beta. Only allow certain people (like regular form goers, or via an application process; random selection of subs) in the beta, and of course they should be "smart" enough to know they are testing a beta.

    The point of the beta is to not only find bugs, but get feedback before launch as well. The objectives of any beta (open or closed) is as thus:

    1. Find the obvious bugs (things that don't work)
    2. Get user feedback on desired and undesired functionality; not bugs (player not able to fast-foward less then 30 sec, layout issues, HD video override; various others)
    3. Get feedback on what users want added to the product
    4. Stress Test
    5. Platform testing of product (to make sure it actually works on PS4, Xbox; the like) as a small Funimation test sample may not capture all issues; testers may not explore all areas
    6. Funi App conflict testing with other user's apps; especially on phones (for example: supposedly some users has RAM recording apps that conflicted with the DRM of the Funi APP, thus breaking)
    7. Market/Branding Testing of the Product (which demographics use it; how they use it, why they use it, why some don't use it)
    8. Build "Hype" for the product… use the beta itself as a marketing tool; the beta users as "representatives" of such (they post on forums/websites how wonderful it is; free adverting for Funimation)

    A beta should only thusly become a final release if it has satisfied the declared business objectives of the product. Or otherwise there will be consequences...

    @Sophie:

    Even with the new apps, users have only reported like 2 or 3 things that we didn't already know about that weren't content issues.

    Thats funny, because was a ton of problems some people were having. Why would any competent Funimation management authorize a release of this product that is still effectively "a beta"? Would it have delayed the product? Of course, but "the sky isn't falling" to get it out now the last time I checked.

    Indeed, the Spring Anime Simulcast season was far too weak to hype up, and delaying FUNimation now until closer to summer (surge the hype at the key time) would have strategically been far more logical. Instead, Funimation "spent all of its bullets" releasing a product in Winter; one that didn't do well at launch and failed to garner self-sustaining interest. It now has lost the ability to "hype" the product (people are now "distrustful" of future releases), and covert into sales/subscriptions. Absolutely horrendous marketing decision, no doubt about it.

    @Sophie:

    All of this is extremely valuable, but isn't necessarily anything to delay a launch over, which is what would have happened.

    The very statement ye made that "beta isn't valuable" is the very proof of my original statement:

    @thegrandalliance:

    …preeeety sure this situation is more of the lack of the software developer/programmer's experiential competence to run a beta, then the subscriber's ability to make use thereof...

    …as if you don't do these things, Funimation is "flying blind" when it releases a product. People who get mad and /fedup and even /ragequit is very possible. True, in long term they should overcome such problems; nonetheless, if Funimation wants the product to come out "clean and beautiful" the first time, it needs to do the hard work beforehand.

    The only reason NOT to have beta is to keep the product "secret", for whatever strategic reason. Otherwise, there is absolutely no logical reason otherwise not to; as else you must accept the consequences of releasing a perceived "bad" product, as it were.

    Bottom Line: You don't release a product before its ready. Or otherwise you will turn people off.

    And if there is enough "complaining" (justified or not) you will even turn off people who otherwise wouldn't twice. They read the forums, get some "vague idea" that "Funimation released a bad product", and their brand image of Funimation is forever modified.

    Alas, I am starting to question the management decisions of Funimation; for this and many other recent issues as it were, such incompetence it is so...

    YOU SHOULD BE WATCHING…
    (your $$$ that is …)



  • I really appreciate you taking the time write all of this. I understand that this comes out of frustration and you would not want to have to tell us something that should be obvious. You were promised an amazing mobile app and were delivered something that was incomplete.

    While I do agree that releasing a product with flaws can and does cause people to never come back and that a Beta does give us a chance to receive user feedback, if we were to delay release until a product is perfect, then it wouldn't be released at all. We certainly wouldn't have the site that we are posting on right now.

    I can certainly suggest a closed beta to the higher ups. At the very least, it would support what the QA team and other stock holders flag as possible issues prior to a wider launch and would help to prioritize those issues.



  • @Sophie:

    if we were to delay release until a product is perfect, then it wouldn't be released at all. We certainly wouldn't have the site that we are posting on right now.

    BTW do not feel like you need to defend anything Sophie. I am not mad at you… rather so at the "rectangular prism heads" who are apparently controlling "Group 1200 Media" (parent of Funimation) at the moment.

    I am not asking to wait forever for it to be perfect. Simply so that it is "more likely then not" to have "a positive affect" on the user. Apparently, it was met with very lukewarm (putting it nicely) reviews. It is simply strategy: Releasing a product you know is gonna have a rough time in Winter, vs waiting until the optimal market timing (midyear) and making sure it is smoother, is what has been lost.

    By releasing it in late winter; yes, you may have "hid" the problems with it. But you also forfeited the ability to "make a splash" with the product. I am not saying you needed to release "a perfect product". but apparently from what users are having problems with, it was too soon. It wasn't simply a "obscure handful of users" having problems... it was widespread across all of the various platforms (Xbox, Windows 10, PS4, Android). Problems that should have (and are now) been dealt with before launch.

    I know Funimation "hyped" the product at the Las Vegas Consumer Electronics Show in January. They promised a timetable. However, they failed to deliver the product people were supposedly "promised"; a smooth and "awesome" Funimation Anime experience. As a result, whatever hype "FuniManagement" was hoping for in January, faded away with this launch.

    Yes, people don't like delays; if you said you gonna release in Feb and didn't come until May, that is one thing. However, they don't like what they perceive as a "bad" product even worse. IDK who's idea was it to launch in February; generally a slow time of year, but it was horrible idea. If you wanted the FunINOW app to carry a marketing message throughout this year (as evidenced by the re-branding of the logo), that was just horrible strategy; unless it was actually "ready" to go.

    What is worse is that the Spring Lineup wasn't up to par (I am personally OK with it, but it isn't as good as Winter). By releasing a so-so app + soso season; any "hype momentum" that you were trying to get just derailed. If you knew Spring was gonna be bad, and you dump a "bad" app, you just = disappointed people. Sediment for Millennials, especially, is a very thickle thing.

    Now, it isn't all bad news: 1 year from now, the App's problems will probably go away, and short term sediment will subside. However, the potential impact Funimation could have had is lost. Worse, there is a "stacking" of problems going on:

    1. Funimation losing long-term license agreements left/right; Aniplex, Kadokawa, others
    2. Funimation losing what are considered "prime" licenses to its rivals; Crunchyroll, Netflix, Amazon

    Now, understanding all of this is going on, you now add:

    3. Funimation has, as observed by "the anime community", a "below average" Spring Simulcast Season
    4. A New App, that for many "doesn't work", or doesn't work to their expectations

    Each of these effects by themselves is "manageable". When you have all 4 occurring in the same quarter or so, that is a problem. The consumers for one, are getting a very real idea that "something larger" is going on; as evidence by these forums or otherwise. When this occurs, it can lead to long-term brand damage.

    ...or potentially very real damage. Anyways, my point is that, there is growing evidence that Funimation is doing "everything humanly possible" to "unwittingly bankrupt the company". Indeed, this is probably a gross hyperbole of the issue... nevertheless, there are publically observable and strategic errors being made in rapid succession that is disconcerting. And as to "the community" who has appreciated Funimaiton's otherwise stellar output of quality, localized content; it is a problem. Will companies such as Cruchyroll and larger ones like Amazon/Netflix displace Funimation? Will Japanese companies simply cut out American companies all together?

    I will admit, not many people care if Funimation goes down or not, that is fine. In either case, as a general measure "health" of the Anime (American) industry, some of us at least "try" to care in either case.

    Indeed, I will likely leave it at that. Otherwise, a very long /walloftext is /incoming nonetheless...



  • I can't say much about those problems since I'm just browsing Funimation from a standard home computer. But a thing I do know from my experience as a former programmer/web developper is that most people making the kind of decisions which thegrandalliance pointed as "bad" (beta/no beta, delivery dates, budget management, and all those things) rarely have any idea of what people under them really do nor do they understand what it is all about.

    So my guess is that all those poor devs and other workers at the bottom of the scale are doing the best they can to meet impossible deadlines made up by people who don't know what they're doing.



  • @the_krock:

    So my guess is that all those poor devs and other workers at the bottom of the scale are doing the best they can to meet impossible deadlines made up by people who don't know what they're doing.

    Exactly. Hense the well-crafted (presidential election inspired) message: "Rectangular Prism Heads" (think about it for a few seconds, you get it).

    In either case, we will have to wait and see what Funimaiton has in store next, it is so…

    WE SHALL BE WATCHING…


Log in to reply